From 2da448b43f3427700e5f59e8f16f507aa2e1f372 Mon Sep 17 00:00:00 2001 From: Alex Auvolat Date: Tue, 24 May 2022 15:28:37 +0200 Subject: Add documentation for new Admin API and a few infos on K2V --- doc/book/reference-manual/admin-api.md | 626 +++++++++++++++++++++++++++++ doc/book/reference-manual/configuration.md | 36 +- doc/book/reference-manual/k2v.md | 58 +++ 3 files changed, 716 insertions(+), 4 deletions(-) create mode 100644 doc/book/reference-manual/admin-api.md create mode 100644 doc/book/reference-manual/k2v.md (limited to 'doc/book/reference-manual') diff --git a/doc/book/reference-manual/admin-api.md b/doc/book/reference-manual/admin-api.md new file mode 100644 index 00000000..a891da07 --- /dev/null +++ b/doc/book/reference-manual/admin-api.md @@ -0,0 +1,626 @@ ++++ +title = "Specification of Garage's administration API" +weight = 16 ++++ + +The Garage administration API is accessible through a dedicated server whose +listen address is specified in the `[admin]` section of the configuration +file (see [configuration file +reference](@/documentation/reference-manual/configuration.md)) + +**WARNING.** At this point, there is no comittement to stability of the APIs described in this document. +We will bump the version numbers prefixed to each API endpoint at each time the syntax +or semantics change, meaning that code that relies on these endpoint will break +when changes are introduced. + +The Garage administration API was introduced in version 0.7.2, this document +does not apply to older versions of Garage. + + +## Access control + +The admin API uses two different tokens for acces control, that are specified in the config file's `[admin]` section: + +- `metrics_token`: the token for accessing the Metrics endpoint (if this token + is not set in the config file, the Metrics endpoint can be accessed without + access control); + +- `admin_token`: the token for accessing all of the other administration + endpoints (if this token is not set in the config file, access to these + endpoints is disabled entirely). + +These tokens are used as simple HTTP bearer tokens. In other words, to +authenticate access to an admin API endpoint, add the following HTTP header +to your request: + +``` +Authorization: Bearer +``` + +## Administration API endpoints + +### Metrics-related endpoints + +#### Metrics `GET /metrics` + +Returns internal Garage metrics in Prometheus format. + +### Cluster operations + +#### GetClusterStatus `GET /v0/status` + +Returns the cluster's current status in JSON, including: + +- ID of the node being queried and its version of the Garage daemon +- Live nodes +- Currently configured cluster layout +- Staged changes to the cluster layout + +Example response body: + +```json +{ + "node": "ec79480e0ce52ae26fd00c9da684e4fa56658d9c64cdcecb094e936de0bfe71f", + "garage_version": "git:v0.8.0", + "knownNodes": { + "ec79480e0ce52ae26fd00c9da684e4fa56658d9c64cdcecb094e936de0bfe71f": { + "addr": "10.0.0.11:3901", + "is_up": true, + "last_seen_secs_ago": 9, + "hostname": "node1" + }, + "4a6ae5a1d0d33bf895f5bb4f0a418b7dc94c47c0dd2eb108d1158f3c8f60b0ff": { + "addr": "10.0.0.12:3901", + "is_up": true, + "last_seen_secs_ago": 1, + "hostname": "node2" + }, + "23ffd0cdd375ebff573b20cc5cef38996b51c1a7d6dbcf2c6e619876e507cf27": { + "addr": "10.0.0.21:3901", + "is_up": true, + "last_seen_secs_ago": 7, + "hostname": "node3" + }, + "e2ee7984ee65b260682086ec70026165903c86e601a4a5a501c1900afe28d84b": { + "addr": "10.0.0.22:3901", + "is_up": true, + "last_seen_secs_ago": 1, + "hostname": "node4" + } + }, + "layout": { + "version": 12, + "roles": { + "ec79480e0ce52ae26fd00c9da684e4fa56658d9c64cdcecb094e936de0bfe71f": { + "zone": "dc1", + "capacity": 4, + "tags": [ + "node1" + ] + }, + "4a6ae5a1d0d33bf895f5bb4f0a418b7dc94c47c0dd2eb108d1158f3c8f60b0ff": { + "zone": "dc1", + "capacity": 6, + "tags": [ + "node2" + ] + }, + "23ffd0cdd375ebff573b20cc5cef38996b51c1a7d6dbcf2c6e619876e507cf27": { + "zone": "dc2", + "capacity": 10, + "tags": [ + "node3" + ] + } + }, + "stagedRoleChanges": { + "e2ee7984ee65b260682086ec70026165903c86e601a4a5a501c1900afe28d84b": { + "zone": "dc2", + "capacity": 5, + "tags": [ + "node4" + ] + } + } + } +} +``` + +#### ConnectClusterNodes `POST /v0/connect` + +Instructs this Garage node to connect to other Garage nodes at specified addresses. + +Example request body: + +```json +[ + "ec79480e0ce52ae26fd00c9da684e4fa56658d9c64cdcecb094e936de0bfe71f@10.0.0.11:3901", + "4a6ae5a1d0d33bf895f5bb4f0a418b7dc94c47c0dd2eb108d1158f3c8f60b0ff@10.0.0.12:3901" +] +``` + +The format of the string for a node to connect to is: `@:`, same as in the `garage node connect` CLI call. + +Example response: + +```json +[ + { + "success": true, + "error": null + }, + { + "success": false, + "error": "Handshake error" + } +] +``` + +#### GetClusterLayout `GET /v0/layout` + +Returns the cluster's current layout in JSON, including: + +- Currently configured cluster layout +- Staged changes to the cluster layout + +(the info returned by this endpoint is a subset of the info returned by GetClusterStatus) + +Example response body: + +```json +{ + "version": 12, + "roles": { + "ec79480e0ce52ae26fd00c9da684e4fa56658d9c64cdcecb094e936de0bfe71f": { + "zone": "dc1", + "capacity": 4, + "tags": [ + "node1" + ] + }, + "4a6ae5a1d0d33bf895f5bb4f0a418b7dc94c47c0dd2eb108d1158f3c8f60b0ff": { + "zone": "dc1", + "capacity": 6, + "tags": [ + "node2" + ] + }, + "23ffd0cdd375ebff573b20cc5cef38996b51c1a7d6dbcf2c6e619876e507cf27": { + "zone": "dc2", + "capacity": 10, + "tags": [ + "node3" + ] + } + }, + "stagedRoleChanges": { + "e2ee7984ee65b260682086ec70026165903c86e601a4a5a501c1900afe28d84b": { + "zone": "dc2", + "capacity": 5, + "tags": [ + "node4" + ] + } + } +} +``` + +#### UpdateClusterLayout `POST /v0/layout` + +Send modifications to the cluster layout. These modifications will +be included in the staged role changes, visible in subsequent calls +of `GetClusterLayout`. Once the set of staged changes is satisfactory, +the user may call `ApplyClusterLayout` to apply the changed changes, +or `Revert ClusterLayout` to clear all of the staged changes in +the layout. + +Request body format: + +```json +{ + : { + "capacity": , + "zone": , + "tags": [ + , + ... + ] + }, + : null, + ... +} +``` + +Contrary to the CLI that may update only a subset of the fields +`capacity`, `zone` and `tags`, when calling this API all of these +values must be specified. + + +#### ApplyClusterLayout `POST /v0/layout/apply` + +Applies to the cluster the layout changes currently registered as +staged layout changes. + +Request body format: + +```json +{ + "version": 13 +} +``` + +Similarly to the CLI, the body must include the version of the new layout +that will be created, which MUST be 1 + the value of the currently +existing layout in the cluster. + +#### RevertClusterLayout `POST /v0/layout/revert` + +Clears all of the staged layout changes. + +Request body format: + +```json +{ + "version": 13 +} +``` + +Reverting the staged changes is done by incrementing the version number +and clearing the contents of the staged change list. +Similarly to the CLI, the body must include the incremented +version number, which MUST be 1 + the value of the currently +existing layout in the cluster. + + +### Access key operations + +#### ListKeys `GET /v0/key` + +Returns all API access keys in the cluster. + +Example response: + +```json +[ + { + "id": "GK31c2f218a2e44f485b94239e", + "name": "test" + }, + { + "id": "GKe10061ac9c2921f09e4c5540", + "name": "test2" + } +] +``` + +#### CreateKey `POST /v0/key` + +Creates a new API access key. + +Request body format: + +```json +{ + "name": "NameOfMyKey" +} +``` + +#### ImportKey `POST /v0/key/import` + +Imports an existing API key. + +Request body format: + +```json +{ + "accessKeyId": "GK31c2f218a2e44f485b94239e", + "secretAccessKey": "b892c0665f0ada8a4755dae98baa3b133590e11dae3bcc1f9d769d67f16c3835", + "name": "NameOfMyKey" +} +``` + +#### GetKeyInfo `GET /v0/key?id=` +#### GetKeyInfo `GET /v0/key?search=` + +Returns information about the requested API access key. + +If `id` is set, the key is looked up using its exact identifier (faster). +If `search` is set, the key is looked up using its name or prefix +of identifier (slower, all keys are enumerated to do this). + +Example response: + +```json +{ + "name": "test", + "accessKeyId": "GK31c2f218a2e44f485b94239e", + "secretAccessKey": "b892c0665f0ada8a4755dae98baa3b133590e11dae3bcc1f9d769d67f16c3835", + "permissions": { + "createBucket": false + }, + "buckets": [ + { + "id": "70dc3bed7fe83a75e46b66e7ddef7d56e65f3c02f9f80b6749fb97eccb5e1033", + "globalAliases": [ + "test2" + ], + "localAliases": [], + "permissions": { + "read": true, + "write": true, + "owner": false + } + }, + { + "id": "d7452a935e663fc1914f3a5515163a6d3724010ce8dfd9e4743ca8be5974f995", + "globalAliases": [ + "test3" + ], + "localAliases": [], + "permissions": { + "read": true, + "write": true, + "owner": false + } + }, + { + "id": "e6a14cd6a27f48684579ec6b381c078ab11697e6bc8513b72b2f5307e25fff9b", + "globalAliases": [], + "localAliases": [ + "test" + ], + "permissions": { + "read": true, + "write": true, + "owner": true + } + }, + { + "id": "96470e0df00ec28807138daf01915cfda2bee8eccc91dea9558c0b4855b5bf95", + "globalAliases": [ + "alex" + ], + "localAliases": [], + "permissions": { + "read": true, + "write": true, + "owner": true + } + } + ] +} +``` + +#### DeleteKey `DELETE /v0/key?id=` + +Deletes an API access key. + +#### UpdateKey `POST /v0/key?id=` + +Updates information about the specified API access key. + +Request body format: + +```json +{ + "name": "NameOfMyKey", + "allow": { + "createBucket": true, + }, + "deny": {} +} +``` + +All fields (`name`, `allow` and `deny`) are optionnal. +If they are present, the corresponding modifications are applied to the key, otherwise nothing is changed. +The possible flags in `allow` and `deny` are: `createBucket`. + + +### Bucket operations + +#### ListBuckets `GET /v0/bucket` + +Returns all storage buckets in the cluster. + +Example response: + +```json +[ + { + "id": "70dc3bed7fe83a75e46b66e7ddef7d56e65f3c02f9f80b6749fb97eccb5e1033", + "globalAliases": [ + "test2" + ], + "localAliases": [] + }, + { + "id": "96470e0df00ec28807138daf01915cfda2bee8eccc91dea9558c0b4855b5bf95", + "globalAliases": [ + "alex" + ], + "localAliases": [] + }, + { + "id": "d7452a935e663fc1914f3a5515163a6d3724010ce8dfd9e4743ca8be5974f995", + "globalAliases": [ + "test3" + ], + "localAliases": [] + }, + { + "id": "e6a14cd6a27f48684579ec6b381c078ab11697e6bc8513b72b2f5307e25fff9b", + "globalAliases": [], + "localAliases": [ + { + "accessKeyId": "GK31c2f218a2e44f485b94239e", + "alias": "test" + } + ] + } +] +``` + +#### GetBucketInfo `GET /v0/bucket?id=` +#### GetBucketInfo `GET /v0/bucket?globalAlias=` + +Returns information about the requested storage bucket. + +If `id` is set, the bucket is looked up using its exact identifier. +If `globalAlias` is set, the bucket is looked up using its global alias. +(both are fast) + +Example response: + +```json +{ + "id": "e6a14cd6a27f48684579ec6b381c078ab11697e6bc8513b72b2f5307e25fff9b", + "globalAliases": [ + "alex" + ], + "keys": [ + { + "accessKeyId": "GK31c2f218a2e44f485b94239e", + "name": "alex", + "permissions": { + "read": true, + "write": true, + "owner": true + }, + "bucketLocalAliases": [ + "test" + ] + } + ] +} +``` + +#### CreateBucket `POST /v0/bucket` + +Creates a new storage bucket. + +Request body format: + +```json +{ + "globalAlias": "NameOfMyBucket" +} +``` + +OR + +```json +{ + "localAlias": { + "accessKeyId": "GK31c2f218a2e44f485b94239e", + "alias": "NameOfMyBucket", + "allow": { + "read": true, + "write": true, + "owner": false + } + } +} +``` + +OR + +```json +{} +``` + +Creates a new bucket, either with a global alias, a local one, +or no alias at all. + +Technically, you can also specify both `globalAlias` and `localAlias` and that would create +two aliases, but I don't see why you would want to do that. + +#### DeleteBucket `DELETE /v0/bucket?id=` + +Deletes a storage bucket. A bucket cannot be deleted if it is not empty. + +Warning: this will delete all aliases associated with the bucket! + +#### PutBucketWebsite `PUT /v0/bucket/website?id=` + +Sets the website configuration for a bucket (this also enables website access for this bucket). + +Request body format: + +```json +{ + "indexDocument": "index.html", + "errorDocument": "404.html" +} +``` + +The field `errorDocument` is optional, if no error document is set a generic error message is displayed when errors happen. + + +#### DeleteBucketWebsite `DELETE /v0/bucket/website?id=` + +Deletes the website configuration for a bucket (disables website access for this bucket). + + +### Operations on permissions for keys on buckets + +#### BucketAllowKey `POST /v0/bucket/allow` + +Allows a key to do read/write/owner operations on a bucket. + +Request body format: + +```json +{ + "bucketId": "e6a14cd6a27f48684579ec6b381c078ab11697e6bc8513b72b2f5307e25fff9b", + "accessKeyId": "GK31c2f218a2e44f485b94239e", + "permissions": { + "read": true, + "write": true, + "owner": true + }, +} +``` + +Flags in `permissions` which have the value `true` will be activated. +Other flags will remain unchanged. + +#### BucketDenyKey `POST /v0/bucket/deny` + +Denies a key from doing read/write/owner operations on a bucket. + +Request body format: + +```json +{ + "bucketId": "e6a14cd6a27f48684579ec6b381c078ab11697e6bc8513b72b2f5307e25fff9b", + "accessKeyId": "GK31c2f218a2e44f485b94239e", + "permissions": { + "read": false, + "write": false, + "owner": true + }, +} +``` + +Flags in `permissions` which have the value `true` will be deactivated. +Other flags will remain unchanged. + + +### Operations on bucket aliases + +#### GlobalAliasBucket `PUT /v0/bucket/alias/global?id=&alias=` + +Empty body. Creates a global alias for a bucket. + +#### GlobalUnaliasBucket `DELETE /v0/bucket/alias/global?id=&alias=` + +Removes a global alias for a bucket. + +#### LocalAliasBucket `PUT /v0/bucket/alias/local?id=&accessKeyId=&alias=` + +Empty body. Creates a local alias for a bucket in the namespace of a specific access key. + +#### LocalUnaliasBucket `DELETE /v0/bucket/alias/local?id=&accessKeyId&alias=` + +Removes a local alias for a bucket in the namespace of a specific access key. + diff --git a/doc/book/reference-manual/configuration.md b/doc/book/reference-manual/configuration.md index bb04650c..65381f46 100644 --- a/doc/book/reference-manual/configuration.md +++ b/doc/book/reference-manual/configuration.md @@ -10,6 +10,7 @@ metadata_dir = "/var/lib/garage/meta" data_dir = "/var/lib/garage/data" block_size = 1048576 +block_manager_background_tranquility = 2 replication_mode = "3" @@ -47,6 +48,8 @@ root_domain = ".web.garage" [admin] api_bind_addr = "0.0.0.0:3903" +metrics_token = "cacce0b2de4bc2d9f5b5fdff551e01ac1496055aed248202d415398987e35f81" +admin_token = "ae8cb40ea7368bbdbb6430af11cca7da833d3458a5f52086f4e805a570fb5c2a" trace_sink = "http://localhost:4317" ``` @@ -84,6 +87,17 @@ files will remain available. This however means that chunks from existing files will not be deduplicated with chunks from newly uploaded files, meaning you might use more storage space that is optimally possible. +### `block_manager_background_tranquility` + +This parameter tunes the activity of the background worker responsible for +resyncing data blocks between nodes. The higher the tranquility value is set, +the more the background worker will wait between iterations, meaning the load +on the system (including network usage between nodes) will be reduced. The +minimal value for this parameter is `0`, where the background worker will +allways work at maximal throughput to resynchronize blocks. The default value +is `2`, where the background worker will try to spend at most 1/3 of its time +working, and 2/3 sleeping in order to reduce system load. + ### `replication_mode` Garage supports the following replication modes: @@ -326,10 +340,24 @@ Garage has a few administration capabilities, in particular to allow remote moni ### `api_bind_addr` If specified, Garage will bind an HTTP server to this port and address, on -which it will listen to requests for administration features. Currently, -this endpoint only exposes Garage metrics in the Prometheus format at -`/metrics`. This endpoint is not authenticated. In the future, bucket and -access key management might be possible by REST calls to this endpoint. +which it will listen to requests for administration features. +See [administration API reference](@/documentation/reference-manual/admin-api.md) to learn more about these features. + +### `metrics_token` (since version 0.7.2) + +The token for accessing the Metrics endpoint. If this token is not set in +the config file, the Metrics endpoint can be accessed without access +control. + +You can use any random string for this value. We recommend generating a random token with `openssl rand -hex 32`. + +### `admin_token` (since version 0.7.2) + +The token for accessing all of the other administration endpoints. If this +token is not set in the config file, access to these endpoints is disabled +entirely. + +You can use any random string for this value. We recommend generating a random token with `openssl rand -hex 32`. ### `trace_sink` diff --git a/doc/book/reference-manual/k2v.md b/doc/book/reference-manual/k2v.md new file mode 100644 index 00000000..742e4309 --- /dev/null +++ b/doc/book/reference-manual/k2v.md @@ -0,0 +1,58 @@ ++++ +title = "K2V" +weight = 30 ++++ + +Starting with version 0.7.2, Garage introduces an optionnal feature, K2V, +which is an alternative storage API designed to help efficiently store +many small values in buckets (in opposition to S3 which is more designed +to store large blobs). + +K2V is currently disabled at compile time in all builds, as the +specification is still subject to changes. To build a Garage version with +K2V, the Cargo feature flag `k2v` must be activated. Special builds with +the `k2v` feature flag enabled can be obtained from our download page under +"Extra builds": such builds can be identified easily as their tag name ends +with `-k2v` (example: `v0.7.2-k2v`). + +The specification of the K2V API can be found +[here](https://git.deuxfleurs.fr/Deuxfleurs/garage/src/branch/k2v/doc/drafts/k2v-spec.md). +This document also includes a high-level overview of K2V's design. + +The K2V API uses AWSv4 signatures for authentification, same as the S3 API. +The AWS region used for signature calculation is always the same as the one +defined for the S3 API in the config file. + +## Enabling and using K2V + +To enable K2V, download and run a build that has the `k2v` feature flag +enabled, or produce one yourself. Then, add the following section to your +configuration file: + +```toml +[k2v_api] +api_bind_addr = ":" +``` + +Please select a port number that is not already in use by another API +endpoint (S3 api, admin API) or by the RPC server. + +We provide an early-stage K2V client library for Rust which can be imported by adding the following to your `Cargo.toml` file: + +```toml +k2v-client = { git = "https://git.deuxfleurs.fr/Deuxfleurs/garage.git" } +``` + +There is also a simple CLI utility which can be built from source in the +following way: + +```sh +git clone https://git.deuxfleurs.fr/Deuxfleurs/garage.git +cd garage/src/k2v-client +cargo build --features cli --bin k2v-cli +``` + +The CLI utility is self-documented, run `k2v-cli --help` to learn how to use +it. There is also a short README.md in the `src/k2v-client` folder with some +instructions. + -- cgit v1.2.3 From 9f303f6308a313c8310c3597ecb6828e46352821 Mon Sep 17 00:00:00 2001 From: Alex Auvolat Date: Tue, 24 May 2022 15:47:42 +0200 Subject: Shorter page title --- doc/book/reference-manual/admin-api.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc/book/reference-manual') diff --git a/doc/book/reference-manual/admin-api.md b/doc/book/reference-manual/admin-api.md index a891da07..b77f0d39 100644 --- a/doc/book/reference-manual/admin-api.md +++ b/doc/book/reference-manual/admin-api.md @@ -1,5 +1,5 @@ +++ -title = "Specification of Garage's administration API" +title = "Administration API" weight = 16 +++ -- cgit v1.2.3 From 43ddc933f9eb36a98369fe671005e35005d8e3cd Mon Sep 17 00:00:00 2001 From: Quentin Dufour Date: Wed, 25 May 2022 15:20:08 +0200 Subject: Update Ceph S3 endpoints compatibility --- doc/book/reference-manual/s3-compatibility.md | 64 +++++++++++++++------------ 1 file changed, 36 insertions(+), 28 deletions(-) (limited to 'doc/book/reference-manual') diff --git a/doc/book/reference-manual/s3-compatibility.md b/doc/book/reference-manual/s3-compatibility.md index 71b4c209..2a0b2ac7 100644 --- a/doc/book/reference-manual/s3-compatibility.md +++ b/doc/book/reference-manual/s3-compatibility.md @@ -3,18 +3,28 @@ title = "S3 Compatibility status" weight = 20 +++ +## DISCLAIMER + +**The compatibility list for other platforms is given only for information purposes and based on available documentation.** They are sometimes completed, in a best effort approach, with the source code and inputs from maintainers when documentation is lacking. We are not proactively monitoring new versions of each software, check the modification history to know when the page has been updated for the last time. Some entries will be inexact or outdated: for any serious decision, you must make your own tests. +**The official documentation of each project can be accessed by clicking on the project name in the column header.** + +Feel free to open a PR to fix this table. Minio is missing because they do not provide a public S3 compatibility list. + +## Update history + + - 2022-02-07 - First version of this page + - 2022-05-25 - Many Ceph S3 endpoints are not documented but implemented. Following a notification from the Ceph community, we added them. + ## Endpoint implementation All APIs that are missing on Garage will return a 501 Not Implemented. Some `x-amz-` headers are not implemented. -*The compatibility list for other platforms is given only for information purposes and based on available documentation. Some entries might be inexact. Feel free to open a PR to fix this table. Minio is missing because they do not provide a public S3 compatibility list.* - ### Features | Feature | Garage | [Openstack Swift](https://docs.openstack.org/swift/latest/s3_compat.html) | [Ceph Object Gateway](https://docs.ceph.com/en/latest/radosgw/s3/) | [Riak CS](https://docs.riak.com/riak/cs/2.1.1/references/apis/storage/s3/index.html) | [OpenIO](https://docs.openio.io/latest/source/arch-design/s3_compliancy.html) | |------------------------------|----------------------------------|-----------------|---------------|---------|-----| -| [signature v2](https://docs.aws.amazon.com/general/latest/gr/signature-version-2.html) (deprecated) | ❌ Missing | ✅ | ❌ | ✅ | ✅ | +| [signature v2](https://docs.aws.amazon.com/general/latest/gr/signature-version-2.html) (deprecated) | ❌ Missing | ✅ | ✅ | ✅ | ✅ | | [signature v4](https://docs.aws.amazon.com/AmazonS3/latest/API/sig-v4-authenticating-requests.html) | ✅ Implemented | ✅ | ✅ | ❌ | ✅ | | [URL path-style](https://docs.aws.amazon.com/AmazonS3/latest/userguide/VirtualHosting.html#path-style-access) (eg. `host.tld/bucket/key`) | ✅ Implemented | ✅ | ✅ | ❓| ✅ | | [URL vhost-style](https://docs.aws.amazon.com/AmazonS3/latest/userguide/VirtualHosting.html#virtual-hosted-style-access) URL (eg. `bucket.host.tld/key`) | ✅ Implemented | ❌| ✅| ✅ | ✅ | @@ -37,7 +47,7 @@ Some `x-amz-` headers are not implemented. | [DeleteObjects](https://docs.aws.amazon.com/AmazonS3/latest/API/API_DeleteObjects.html) | ✅ Implemented | ✅ | ✅ | ✅ | ✅ | | [GetObject](https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObject.html) | ✅ Implemented | ✅ | ✅ | ✅ | ✅ | | [ListObjects](https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListObjects.html) | ✅ Implemented (see details below) | ✅ | ✅ | ✅ | ❌| -| [ListObjectsV2](https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListObjectsV2.html) | ✅ Implemented | ❌| ❌| ❌| ✅ | +| [ListObjectsV2](https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListObjectsV2.html) | ✅ Implemented | ❌| ✅ | ❌| ✅ | | [PostObject](https://docs.aws.amazon.com/AmazonS3/latest/API/RESTObjectPOST.html) (compatibility API) | ❌ Missing | ❌| ✅ | ❌| ❌| | [PutObject](https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutObject.html) | ✅ Implemented | ✅ | ✅ | ✅ | ✅ | @@ -67,9 +77,9 @@ For more information, please refer to our [issue tracker](https://git.deuxfleurs | [DeleteBucketWebsite](https://docs.aws.amazon.com/AmazonS3/latest/API/API_DeleteBucketWebsite.html) | ✅ Implemented | ❌| ❌| ❌| ❌| | [GetBucketWebsite](https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetBucketWebsite.html) | ✅ Implemented | ❌ | ❌| ❌| ❌| | [PutBucketWebsite](https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutBucketWebsite.html) | ⚠ Partially implemented (see below)| ❌| ❌| ❌| ❌| -| [DeleteBucketCors](https://docs.aws.amazon.com/AmazonS3/latest/API/API_DeleteBucketCors.html) | ✅ Implemented | ❌| ❌| ❌| ✅ | -| [GetBucketCors](https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetBucketCors.html) | ✅ Implemented | ❌ | ❌| ❌| ✅ | -| [PutBucketCors](https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutBucketCors.html) | ✅ Implemented | ❌| ❌| ❌| ✅ | +| [DeleteBucketCors](https://docs.aws.amazon.com/AmazonS3/latest/API/API_DeleteBucketCors.html) | ✅ Implemented | ❌| ✅ | ❌| ✅ | +| [GetBucketCors](https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetBucketCors.html) | ✅ Implemented | ❌ | ✅ | ❌| ✅ | +| [PutBucketCors](https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutBucketCors.html) | ✅ Implemented | ❌| ✅ | ❌| ✅ | **PutBucketWebsite:** Implemented, but only stores the index document suffix and the error document path. Redirects are not supported. @@ -83,16 +93,16 @@ See Garage CLI reference manual to learn how to use Garage's permission system. | Endpoint | Garage | [Openstack Swift](https://docs.openstack.org/swift/latest/s3_compat.html) | [Ceph Object Gateway](https://docs.ceph.com/en/latest/radosgw/s3/) | [Riak CS](https://docs.riak.com/riak/cs/2.1.1/references/apis/storage/s3/index.html) | [OpenIO](https://docs.openio.io/latest/source/arch-design/s3_compliancy.html) | |------------------------------|----------------------------------|-----------------|---------------|---------|-----| -| [DeleteBucketPolicy](https://docs.aws.amazon.com/AmazonS3/latest/API/API_DeleteBucketPolicy.html) | ❌ Missing | ❌| ❌| ✅ | ❌| -| [GetBucketPolicy](https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetBucketPolicy.html) | ❌ Missing | ❌| ❌| ⚠ | ❌| -| [GetBucketPolicyStatus](https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetBucketPolicyStatus.html) | ❌ Missing | ❌| ❌| ❌| ❌| -| [PutBucketPolicy](https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutBucketPolicy.html) | ❌ Missing | ❌| ❌| ⚠ | ❌| +| [DeleteBucketPolicy](https://docs.aws.amazon.com/AmazonS3/latest/API/API_DeleteBucketPolicy.html) | ❌ Missing | ❌| ✅ | ✅ | ❌| +| [GetBucketPolicy](https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetBucketPolicy.html) | ❌ Missing | ❌| ✅ | ⚠ | ❌| +| [GetBucketPolicyStatus](https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetBucketPolicyStatus.html) | ❌ Missing | ❌| ✅ | ❌| ❌| +| [PutBucketPolicy](https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutBucketPolicy.html) | ❌ Missing | ❌| ✅ | ⚠ | ❌| | [GetBucketAcl](https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetBucketAcl.html) | ❌ Missing | ✅ | ✅ | ✅ | ✅ | | [PutBucketAcl](https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutBucketAcl.html) | ❌ Missing | ✅ | ✅ | ✅ | ✅ | | [GetObjectAcl](https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObjectAcl.html) | ❌ Missing | ✅ | ✅ | ✅ | ✅ | | [PutObjectAcl](https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutObjectAcl.html) | ❌ Missing | ✅ | ✅ | ✅ | ✅ | -*Notes:* Ceph claims that it supports bucket policies but does not implement any Policy endpoints. They probably refer to their own permission system. Riak CS only supports a subset of the policy configuration. +*Notes:* Riak CS only supports a subset of the policy configuration. ### Versioning, Lifecycle endpoints @@ -102,8 +112,8 @@ If you need this feature, please [share your use case in our dedicated issue](ht | Endpoint | Garage | [Openstack Swift](https://docs.openstack.org/swift/latest/s3_compat.html) | [Ceph Object Gateway](https://docs.ceph.com/en/latest/radosgw/s3/) | [Riak CS](https://docs.riak.com/riak/cs/2.1.1/references/apis/storage/s3/index.html) | [OpenIO](https://docs.openio.io/latest/source/arch-design/s3_compliancy.html) | |------------------------------|----------------------------------|-----------------|---------------|---------|-----| | [DeleteBucketLifecycle](https://docs.aws.amazon.com/AmazonS3/latest/API/API_DeleteBucketLifecycle.html) | ❌ Missing | ❌| ✅| ❌| ✅| -| [GetBucketLifecycleConfiguration](https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetBucketLifecycleConfiguration.html) | ❌ Missing | ❌| ⚠ | ❌| ✅| -| [PutBucketLifecycleConfiguration](https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutBucketLifecycleConfiguration.html) | ❌ Missing | ❌| ⚠ | ❌| ✅| +| [GetBucketLifecycleConfiguration](https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetBucketLifecycleConfiguration.html) | ❌ Missing | ❌| ✅ | ❌| ✅| +| [PutBucketLifecycleConfiguration](https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutBucketLifecycleConfiguration.html) | ❌ Missing | ❌| ✅ | ❌| ✅| | [GetBucketVersioning](https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetBucketVersioning.html) | ❌ Stub (see below) | ✅| ✅ | ❌| ✅| | [ListObjectVersions](https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListObjectVersions.html) | ❌ Missing | ❌| ✅ | ❌| ✅| | [PutBucketVersioning](https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutBucketVersioning.html) | ❌ Missing | ❌| ✅| ❌| ✅| @@ -111,8 +121,6 @@ If you need this feature, please [share your use case in our dedicated issue](ht **GetBucketVersioning:** Stub implementation (Garage does not yet support versionning so this always returns "versionning not enabled"). -*Note: Ceph only supports `Expiration`, `NoncurrentVersionExpiration` and `AbortIncompleteMultipartUpload` on its Lifecycle endpoints.* - ### Replication endpoints Please open an issue if you have a use case for replication. @@ -135,8 +143,8 @@ Amazon defines a concept of [object locking](https://docs.aws.amazon.com/AmazonS | [PutObjectLegalHold](https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutObjectLegalHold.html) | ❌ Missing | ❌| ✅ | ❌| ❌| | [GetObjectRetention](https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObjectRetention.html) | ❌ Missing | ❌| ✅ | ❌| ❌| | [PutObjectRetention](https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutObjectRetention.html) | ❌ Missing | ❌| ✅ | ❌| ❌| -| [GetObjectLockConfiguration](https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObjectLockConfiguration.html) | ❌ Missing | ❌| ❌| ❌| ❌| -| [PutObjectLockConfiguration](https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutObjectLockConfiguration.html) | ❌ Missing | ❌| ❌| ❌| ❌| +| [GetObjectLockConfiguration](https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObjectLockConfiguration.html) | ❌ Missing | ❌| ✅ | ❌| ❌| +| [PutObjectLockConfiguration](https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutObjectLockConfiguration.html) | ❌ Missing | ❌| ✅ | ❌| ❌| ### (Server-side) encryption @@ -145,9 +153,9 @@ Please open an issue if you have a use case. | Endpoint | Garage | [Openstack Swift](https://docs.openstack.org/swift/latest/s3_compat.html) | [Ceph Object Gateway](https://docs.ceph.com/en/latest/radosgw/s3/) | [Riak CS](https://docs.riak.com/riak/cs/2.1.1/references/apis/storage/s3/index.html) | [OpenIO](https://docs.openio.io/latest/source/arch-design/s3_compliancy.html) | |------------------------------|----------------------------------|-----------------|---------------|---------|-----| -| [DeleteBucketEncryption](https://docs.aws.amazon.com/AmazonS3/latest/API/API_DeleteBucketEncryption.html) | ❌ Missing | ❌| ❌| ❌| ❌| -| [GetBucketEncryption](https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetBucketEncryption.html) | ❌ Missing | ❌| ❌| ❌| ❌| -| [PutBucketEncryption](https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutBucketEncryption.html) | ❌ Missing | ❌| ❌| ❌| ❌| +| [DeleteBucketEncryption](https://docs.aws.amazon.com/AmazonS3/latest/API/API_DeleteBucketEncryption.html) | ❌ Missing | ❌| ✅ | ❌| ❌| +| [GetBucketEncryption](https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetBucketEncryption.html) | ❌ Missing | ❌| ✅ | ❌| ❌| +| [PutBucketEncryption](https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutBucketEncryption.html) | ❌ Missing | ❌| ✅ | ❌| ❌| ### Misc endpoints @@ -155,13 +163,13 @@ Please open an issue if you have a use case. |------------------------------|----------------------------------|-----------------|---------------|---------|-----| | [GetBucketNotificationConfiguration](https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetBucketNotificationConfiguration.html) | ❌ Missing | ❌| ✅ | ❌| ❌| | [PutBucketNotificationConfiguration](https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutBucketNotificationConfiguration.html) | ❌ Missing | ❌| ✅ | ❌| ❌| -| [DeleteBucketTagging](https://docs.aws.amazon.com/AmazonS3/latest/API/API_DeleteBucketTagging.html) | ❌ Missing | ❌| ❌| ❌| ✅ | -| [GetBucketTagging](https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetBucketTagging.html) | ❌ Missing | ❌| ❌| ❌| ✅ | -| [PutBucketTagging](https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutBucketTagging.html) | ❌ Missing | ❌| ❌| ❌| ✅ | -| [DeleteObjectTagging](https://docs.aws.amazon.com/AmazonS3/latest/API/API_DeleteObjectTagging.html) | ❌ Missing | ❌| ❌| ❌| ✅ | -| [GetObjectTagging](https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObjectTagging.html) | ❌ Missing | ❌| ❌| ❌| ✅ | -| [PutObjectTagging](https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutObjectTagging.html) | ❌ Missing | ❌| ❌| ❌| ✅ | -| [GetObjectTorrent](https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObjectTorrent.html) | ❌ Missing | ❌| ❌| ❌| ❌| +| [DeleteBucketTagging](https://docs.aws.amazon.com/AmazonS3/latest/API/API_DeleteBucketTagging.html) | ❌ Missing | ❌| ✅ | ❌| ✅ | +| [GetBucketTagging](https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetBucketTagging.html) | ❌ Missing | ❌| ✅ | ❌| ✅ | +| [PutBucketTagging](https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutBucketTagging.html) | ❌ Missing | ❌| ✅ | ❌| ✅ | +| [DeleteObjectTagging](https://docs.aws.amazon.com/AmazonS3/latest/API/API_DeleteObjectTagging.html) | ❌ Missing | ❌| ✅ | ❌| ✅ | +| [GetObjectTagging](https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObjectTagging.html) | ❌ Missing | ❌| ✅ | ❌| ✅ | +| [PutObjectTagging](https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutObjectTagging.html) | ❌ Missing | ❌| ✅ | ❌| ✅ | +| [GetObjectTorrent](https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObjectTorrent.html) | ❌ Missing | ❌| ✅ | ❌| ❌| ### Vendor specific endpoints -- cgit v1.2.3 From 93eab8eaa3927b99626fee4a747a0f9f041cafdb Mon Sep 17 00:00:00 2001 From: Alex Date: Wed, 25 May 2022 16:54:44 +0200 Subject: Fixes to S3 compatibility page (#314) Mention PostObject is implemented, fix english mistakes Co-authored-by: Alex Auvolat Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/314 Co-authored-by: Alex Co-committed-by: Alex --- doc/book/reference-manual/s3-compatibility.md | 77 +++++++++++++++++---------- 1 file changed, 48 insertions(+), 29 deletions(-) (limited to 'doc/book/reference-manual') diff --git a/doc/book/reference-manual/s3-compatibility.md b/doc/book/reference-manual/s3-compatibility.md index 2a0b2ac7..a8e503d5 100644 --- a/doc/book/reference-manual/s3-compatibility.md +++ b/doc/book/reference-manual/s3-compatibility.md @@ -5,22 +5,26 @@ weight = 20 ## DISCLAIMER -**The compatibility list for other platforms is given only for information purposes and based on available documentation.** They are sometimes completed, in a best effort approach, with the source code and inputs from maintainers when documentation is lacking. We are not proactively monitoring new versions of each software, check the modification history to know when the page has been updated for the last time. Some entries will be inexact or outdated: for any serious decision, you must make your own tests. -**The official documentation of each project can be accessed by clicking on the project name in the column header.** - -Feel free to open a PR to fix this table. Minio is missing because they do not provide a public S3 compatibility list. +**The compatibility list for other platforms is given only for informational +purposes and based on available documentation.** They are sometimes completed, +in a best effort approach, with the source code and inputs from maintainers +when documentation is lacking. We are not proactively monitoring new versions +of each software: check the modification history to know when the page has been +updated for the last time. Some entries will be inexact or outdated. For any +serious decision, you must make your own tests. +**The official documentation of each project can be accessed by clicking on the +project name in the column header.** + +Feel free to open a PR to suggest fixes this table. Minio is missing because they do not provide a public S3 compatibility list. ## Update history - - 2022-02-07 - First version of this page - - 2022-05-25 - Many Ceph S3 endpoints are not documented but implemented. Following a notification from the Ceph community, we added them. +- 2022-02-07 - First version of this page +- 2022-05-25 - Many Ceph S3 endpoints are not documented but implemented. Following a notification from the Ceph community, we added them. -## Endpoint implementation -All APIs that are missing on Garage will return a 501 Not Implemented. -Some `x-amz-` headers are not implemented. -### Features +## High-level features | Feature | Garage | [Openstack Swift](https://docs.openstack.org/swift/latest/s3_compat.html) | [Ceph Object Gateway](https://docs.ceph.com/en/latest/radosgw/s3/) | [Riak CS](https://docs.riak.com/riak/cs/2.1.1/references/apis/storage/s3/index.html) | [OpenIO](https://docs.openio.io/latest/source/arch-design/s3_compliancy.html) | |------------------------------|----------------------------------|-----------------|---------------|---------|-----| @@ -30,34 +34,46 @@ Some `x-amz-` headers are not implemented. | [URL vhost-style](https://docs.aws.amazon.com/AmazonS3/latest/userguide/VirtualHosting.html#virtual-hosted-style-access) URL (eg. `bucket.host.tld/key`) | ✅ Implemented | ❌| ✅| ✅ | ✅ | | [Presigned URLs](https://docs.aws.amazon.com/AmazonS3/latest/userguide/ShareObjectPreSignedURL.html) | ✅ Implemented | ❌| ✅ | ✅ | ✅(❓) | -*Note:* OpenIO does not says if it supports presigned URLs. Because it is part of signature v4 and they claim they support it without additional precisions, we suppose that OpenIO supports presigned URLs. +*Note:* OpenIO does not says if it supports presigned URLs. Because it is part +of signature v4 and they claim they support it without additional precisions, +we suppose that OpenIO supports presigned URLs. + + +## Endpoint implementation + +All endpoints that are missing on Garage will return a 501 Not Implemented. +Some `x-amz-` headers are not implemented. ### Core endoints -| Endpoint | Garage | [Openstack Swift](https://docs.openstack.org/swift/latest/s3_compat.html) | [Ceph Object Gateway](https://docs.ceph.com/en/latest/radosgw/s3/) | [Riak CS](https://docs.riak.com/riak/cs/2.1.1/references/apis/storage/s3/index.html) | [OpenIO](https://docs.openio.io/latest/source/arch-design/s3_compliancy.html) | +| Endpoint | Garage | [Openstack Swift](https://docs.openstack.org/swift/latest/s3_compat.html) | [Ceph Object Gateway](https://docs.ceph.com/en/latest/radosgw/s3/) | [Riak CS](https://docs.riak.com/riak/cs/2.1.1/references/apis/storage/s3/index.html) | [OpenIO](https://docs.openio.io/latest/source/arch-design/s3_compliancy.html) | |------------------------------|----------------------------------|-----------------|---------------|---------|-----| | [CreateBucket](https://docs.aws.amazon.com/AmazonS3/latest/API/API_CreateBucket.html) | ✅ Implemented | ✅ | ✅ | ✅ | ✅ | | [DeleteBucket](https://docs.aws.amazon.com/AmazonS3/latest/API/API_DeleteBucket.html) | ✅ Implemented | ✅ | ✅ | ✅ | ✅ | | [GetBucketLocation](https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetBucketLocation.html) | ✅ Implemented | ✅ | ✅ | ❌ | ✅ | | [HeadBucket](https://docs.aws.amazon.com/AmazonS3/latest/API/API_HeadBucket.html) | ✅ Implemented | ✅ | ✅ | ✅ | ✅ | | [ListBuckets](https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListBuckets.html) | ✅ Implemented | ❌| ✅ | ✅ | ✅ | -| [HeadObject](https://docs.aws.amazon.com/AmazonS3/latest/API/API_HeadObject.html) | ✅ Implemented | ✅ | ✅ | ✅ | ✅ | -| [CopyObject](https://docs.aws.amazon.com/AmazonS3/latest/API/API_CopyObject.html) | ✅ Implemented | ✅ | ✅ | ✅ | ✅ | +| [HeadObject](https://docs.aws.amazon.com/AmazonS3/latest/API/API_HeadObject.html) | ✅ Implemented | ✅ | ✅ | ✅ | ✅ | +| [CopyObject](https://docs.aws.amazon.com/AmazonS3/latest/API/API_CopyObject.html) | ✅ Implemented | ✅ | ✅ | ✅ | ✅ | | [DeleteObject](https://docs.aws.amazon.com/AmazonS3/latest/API/API_DeleteObject.html) | ✅ Implemented | ✅ | ✅ | ✅ | ✅ | | [DeleteObjects](https://docs.aws.amazon.com/AmazonS3/latest/API/API_DeleteObjects.html) | ✅ Implemented | ✅ | ✅ | ✅ | ✅ | | [GetObject](https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObject.html) | ✅ Implemented | ✅ | ✅ | ✅ | ✅ | | [ListObjects](https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListObjects.html) | ✅ Implemented (see details below) | ✅ | ✅ | ✅ | ❌| | [ListObjectsV2](https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListObjectsV2.html) | ✅ Implemented | ❌| ✅ | ❌| ✅ | -| [PostObject](https://docs.aws.amazon.com/AmazonS3/latest/API/RESTObjectPOST.html) (compatibility API) | ❌ Missing | ❌| ✅ | ❌| ❌| +| [PostObject](https://docs.aws.amazon.com/AmazonS3/latest/API/RESTObjectPOST.html) | ✅ Implemented | ❌| ✅ | ❌| ❌| | [PutObject](https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutObject.html) | ✅ Implemented | ✅ | ✅ | ✅ | ✅ | -**ListObjects:** Implemented, but there isn't a very good specification of what `encoding-type=url` covers so there might be some encoding bugs. In our implementation the url-encoded fields are in the same in ListObjects as they are in ListObjectsV2. +**ListObjects:** Implemented, but there isn't a very good specification of what +`encoding-type=url` covers so there might be some encoding bugs. In our +implementation the url-encoded fields are in the same in ListObjects as they +are in ListObjectsV2. -*Note: Ceph API documentation is incomplete and miss at least HeadBucket and UploadPartCopy, but these endpoints are documented in [Red Hat Ceph Storage - Chapter 2. Ceph Object Gateway and the S3 API](https://access.redhat.com/documentation/en-us/red_hat_ceph_storage/4/html/developer_guide/ceph-object-gateway-and-the-s3-api)* +*Note: Ceph API documentation is incomplete and lacks at least HeadBucket and UploadPartCopy, +but these endpoints are documented in [Red Hat Ceph Storage - Chapter 2. Ceph Object Gateway and the S3 API](https://access.redhat.com/documentation/en-us/red_hat_ceph_storage/4/html/developer_guide/ceph-object-gateway-and-the-s3-api)* ### Multipart Upload endpoints -| Endpoint | Garage | [Openstack Swift](https://docs.openstack.org/swift/latest/s3_compat.html) | [Ceph Object Gateway](https://docs.ceph.com/en/latest/radosgw/s3/) | [Riak CS](https://docs.riak.com/riak/cs/2.1.1/references/apis/storage/s3/index.html) | [OpenIO](https://docs.openio.io/latest/source/arch-design/s3_compliancy.html) | +| Endpoint | Garage | [Openstack Swift](https://docs.openstack.org/swift/latest/s3_compat.html) | [Ceph Object Gateway](https://docs.ceph.com/en/latest/radosgw/s3/) | [Riak CS](https://docs.riak.com/riak/cs/2.1.1/references/apis/storage/s3/index.html) | [OpenIO](https://docs.openio.io/latest/source/arch-design/s3_compliancy.html) | |------------------------------|----------------------------------|-----------------|---------------|---------|-----| | [AbortMultipartUpload](https://docs.aws.amazon.com/AmazonS3/latest/API/API_AbortMultipartUpload.html) | ✅ Implemented | ✅ | ✅ | ✅ | ✅ | | [CompleteMultipartUpload](https://docs.aws.amazon.com/AmazonS3/latest/API/API_CompleteMultipartUpload.html) | ✅ Implemented (see details below) | ✅ | ✅ | ✅ | ✅ | @@ -72,7 +88,7 @@ For more information, please refer to our [issue tracker](https://git.deuxfleurs ### Website endpoints -| Endpoint | Garage | [Openstack Swift](https://docs.openstack.org/swift/latest/s3_compat.html) | [Ceph Object Gateway](https://docs.ceph.com/en/latest/radosgw/s3/) | [Riak CS](https://docs.riak.com/riak/cs/2.1.1/references/apis/storage/s3/index.html) | [OpenIO](https://docs.openio.io/latest/source/arch-design/s3_compliancy.html) | +| Endpoint | Garage | [Openstack Swift](https://docs.openstack.org/swift/latest/s3_compat.html) | [Ceph Object Gateway](https://docs.ceph.com/en/latest/radosgw/s3/) | [Riak CS](https://docs.riak.com/riak/cs/2.1.1/references/apis/storage/s3/index.html) | [OpenIO](https://docs.openio.io/latest/source/arch-design/s3_compliancy.html) | |------------------------------|----------------------------------|-----------------|---------------|---------|-----| | [DeleteBucketWebsite](https://docs.aws.amazon.com/AmazonS3/latest/API/API_DeleteBucketWebsite.html) | ✅ Implemented | ❌| ❌| ❌| ❌| | [GetBucketWebsite](https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetBucketWebsite.html) | ✅ Implemented | ❌ | ❌| ❌| ❌| @@ -83,7 +99,7 @@ For more information, please refer to our [issue tracker](https://git.deuxfleurs **PutBucketWebsite:** Implemented, but only stores the index document suffix and the error document path. Redirects are not supported. -*Note: Ceph radosgw has some support for static websites but it is different from Amazon one plus it does not implement its configuration endpoints.* +*Note: Ceph radosgw has some support for static websites but it is different from the Amazon one. It also does not implement its configuration endpoints.* ### ACL, Policies endpoints @@ -91,7 +107,7 @@ Amazon has 2 access control mechanisms in S3: ACL (legacy) and policies (new one Garage implements none of them, and has its own system instead, built around a per-access-key-per-bucket logic. See Garage CLI reference manual to learn how to use Garage's permission system. -| Endpoint | Garage | [Openstack Swift](https://docs.openstack.org/swift/latest/s3_compat.html) | [Ceph Object Gateway](https://docs.ceph.com/en/latest/radosgw/s3/) | [Riak CS](https://docs.riak.com/riak/cs/2.1.1/references/apis/storage/s3/index.html) | [OpenIO](https://docs.openio.io/latest/source/arch-design/s3_compliancy.html) | +| Endpoint | Garage | [Openstack Swift](https://docs.openstack.org/swift/latest/s3_compat.html) | [Ceph Object Gateway](https://docs.ceph.com/en/latest/radosgw/s3/) | [Riak CS](https://docs.riak.com/riak/cs/2.1.1/references/apis/storage/s3/index.html) | [OpenIO](https://docs.openio.io/latest/source/arch-design/s3_compliancy.html) | |------------------------------|----------------------------------|-----------------|---------------|---------|-----| | [DeleteBucketPolicy](https://docs.aws.amazon.com/AmazonS3/latest/API/API_DeleteBucketPolicy.html) | ❌ Missing | ❌| ✅ | ✅ | ❌| | [GetBucketPolicy](https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetBucketPolicy.html) | ❌ Missing | ❌| ✅ | ⚠ | ❌| @@ -106,10 +122,10 @@ See Garage CLI reference manual to learn how to use Garage's permission system. ### Versioning, Lifecycle endpoints -Garage does not support (yet) object versioning. +Garage does not (yet) support object versioning. If you need this feature, please [share your use case in our dedicated issue](https://git.deuxfleurs.fr/Deuxfleurs/garage/issues/166). -| Endpoint | Garage | [Openstack Swift](https://docs.openstack.org/swift/latest/s3_compat.html) | [Ceph Object Gateway](https://docs.ceph.com/en/latest/radosgw/s3/) | [Riak CS](https://docs.riak.com/riak/cs/2.1.1/references/apis/storage/s3/index.html) | [OpenIO](https://docs.openio.io/latest/source/arch-design/s3_compliancy.html) | +| Endpoint | Garage | [Openstack Swift](https://docs.openstack.org/swift/latest/s3_compat.html) | [Ceph Object Gateway](https://docs.ceph.com/en/latest/radosgw/s3/) | [Riak CS](https://docs.riak.com/riak/cs/2.1.1/references/apis/storage/s3/index.html) | [OpenIO](https://docs.openio.io/latest/source/arch-design/s3_compliancy.html) | |------------------------------|----------------------------------|-----------------|---------------|---------|-----| | [DeleteBucketLifecycle](https://docs.aws.amazon.com/AmazonS3/latest/API/API_DeleteBucketLifecycle.html) | ❌ Missing | ❌| ✅| ❌| ✅| | [GetBucketLifecycleConfiguration](https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetBucketLifecycleConfiguration.html) | ❌ Missing | ❌| ✅ | ❌| ✅| @@ -125,19 +141,22 @@ If you need this feature, please [share your use case in our dedicated issue](ht Please open an issue if you have a use case for replication. -| Endpoint | Garage | [Openstack Swift](https://docs.openstack.org/swift/latest/s3_compat.html) | [Ceph Object Gateway](https://docs.ceph.com/en/latest/radosgw/s3/) | [Riak CS](https://docs.riak.com/riak/cs/2.1.1/references/apis/storage/s3/index.html) | [OpenIO](https://docs.openio.io/latest/source/arch-design/s3_compliancy.html) | +| Endpoint | Garage | [Openstack Swift](https://docs.openstack.org/swift/latest/s3_compat.html) | [Ceph Object Gateway](https://docs.ceph.com/en/latest/radosgw/s3/) | [Riak CS](https://docs.riak.com/riak/cs/2.1.1/references/apis/storage/s3/index.html) | [OpenIO](https://docs.openio.io/latest/source/arch-design/s3_compliancy.html) | |------------------------------|----------------------------------|-----------------|---------------|---------|-----| | [DeleteBucketReplication](https://docs.aws.amazon.com/AmazonS3/latest/API/API_DeleteBucketReplication.html) | ❌ Missing | ❌| ✅ | ❌| ❌| | [GetBucketReplication](https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetBucketReplication.html) | ❌ Missing | ❌| ✅ | ❌| ❌| | [PutBucketReplication](https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutBucketReplication.html) | ❌ Missing | ❌| ⚠ | ❌| ❌| -*Note: Ceph documentation briefly says that Ceph supports [replication though the S3 API](https://docs.ceph.com/en/latest/radosgw/multisite-sync-policy/#s3-replication-api) but with some limitations. Additionaly, replication endpoints are not documented in the S3 compatibility page so I don't know what kind of support we can expect.* +*Note: Ceph documentation briefly says that Ceph supports +[replication though the S3 API](https://docs.ceph.com/en/latest/radosgw/multisite-sync-policy/#s3-replication-api) +but with some limitations. +Additionaly, replication endpoints are not documented in the S3 compatibility page so I don't know what kind of support we can expect.* ### Locking objects Amazon defines a concept of [object locking](https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-lock.html) that can be achieved either through a Retention period or a Legal hold. -| Endpoint | Garage | [Openstack Swift](https://docs.openstack.org/swift/latest/s3_compat.html) | [Ceph Object Gateway](https://docs.ceph.com/en/latest/radosgw/s3/) | [Riak CS](https://docs.riak.com/riak/cs/2.1.1/references/apis/storage/s3/index.html) | [OpenIO](https://docs.openio.io/latest/source/arch-design/s3_compliancy.html) | +| Endpoint | Garage | [Openstack Swift](https://docs.openstack.org/swift/latest/s3_compat.html) | [Ceph Object Gateway](https://docs.ceph.com/en/latest/radosgw/s3/) | [Riak CS](https://docs.riak.com/riak/cs/2.1.1/references/apis/storage/s3/index.html) | [OpenIO](https://docs.openio.io/latest/source/arch-design/s3_compliancy.html) | |------------------------------|----------------------------------|-----------------|---------------|---------|-----| | [GetObjectLegalHold](https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObjectLegalHold.html) | ❌ Missing | ❌| ✅ | ❌| ❌| | [PutObjectLegalHold](https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutObjectLegalHold.html) | ❌ Missing | ❌| ✅ | ❌| ❌| @@ -151,7 +170,7 @@ Amazon defines a concept of [object locking](https://docs.aws.amazon.com/AmazonS We think that you can either encrypt your server partition or do client-side encryption, so we did not implement server-side encryption for Garage. Please open an issue if you have a use case. -| Endpoint | Garage | [Openstack Swift](https://docs.openstack.org/swift/latest/s3_compat.html) | [Ceph Object Gateway](https://docs.ceph.com/en/latest/radosgw/s3/) | [Riak CS](https://docs.riak.com/riak/cs/2.1.1/references/apis/storage/s3/index.html) | [OpenIO](https://docs.openio.io/latest/source/arch-design/s3_compliancy.html) | +| Endpoint | Garage | [Openstack Swift](https://docs.openstack.org/swift/latest/s3_compat.html) | [Ceph Object Gateway](https://docs.ceph.com/en/latest/radosgw/s3/) | [Riak CS](https://docs.riak.com/riak/cs/2.1.1/references/apis/storage/s3/index.html) | [OpenIO](https://docs.openio.io/latest/source/arch-design/s3_compliancy.html) | |------------------------------|----------------------------------|-----------------|---------------|---------|-----| | [DeleteBucketEncryption](https://docs.aws.amazon.com/AmazonS3/latest/API/API_DeleteBucketEncryption.html) | ❌ Missing | ❌| ✅ | ❌| ❌| | [GetBucketEncryption](https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetBucketEncryption.html) | ❌ Missing | ❌| ✅ | ❌| ❌| @@ -159,7 +178,7 @@ Please open an issue if you have a use case. ### Misc endpoints -| Endpoint | Garage | [Openstack Swift](https://docs.openstack.org/swift/latest/s3_compat.html) | [Ceph Object Gateway](https://docs.ceph.com/en/latest/radosgw/s3/) | [Riak CS](https://docs.riak.com/riak/cs/2.1.1/references/apis/storage/s3/index.html) | [OpenIO](https://docs.openio.io/latest/source/arch-design/s3_compliancy.html) | +| Endpoint | Garage | [Openstack Swift](https://docs.openstack.org/swift/latest/s3_compat.html) | [Ceph Object Gateway](https://docs.ceph.com/en/latest/radosgw/s3/) | [Riak CS](https://docs.riak.com/riak/cs/2.1.1/references/apis/storage/s3/index.html) | [OpenIO](https://docs.openio.io/latest/source/arch-design/s3_compliancy.html) | |------------------------------|----------------------------------|-----------------|---------------|---------|-----| | [GetBucketNotificationConfiguration](https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetBucketNotificationConfiguration.html) | ❌ Missing | ❌| ✅ | ❌| ❌| | [PutBucketNotificationConfiguration](https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutBucketNotificationConfiguration.html) | ❌ Missing | ❌| ✅ | ❌| ❌| @@ -176,7 +195,7 @@ Please open an issue if you have a use case.
Display Amazon specifc endpoints -| Endpoint | Garage | [Openstack Swift](https://docs.openstack.org/swift/latest/s3_compat.html) | [Ceph Object Gateway](https://docs.ceph.com/en/latest/radosgw/s3/) | [Riak CS](https://docs.riak.com/riak/cs/2.1.1/references/apis/storage/s3/index.html) | [OpenIO](https://docs.openio.io/latest/source/arch-design/s3_compliancy.html) | +| Endpoint | Garage | [Openstack Swift](https://docs.openstack.org/swift/latest/s3_compat.html) | [Ceph Object Gateway](https://docs.ceph.com/en/latest/radosgw/s3/) | [Riak CS](https://docs.riak.com/riak/cs/2.1.1/references/apis/storage/s3/index.html) | [OpenIO](https://docs.openio.io/latest/source/arch-design/s3_compliancy.html) | |------------------------------|----------------------------------|-----------------|---------------|---------|-----| | [DeleteBucketAnalyticsConfiguration](https://docs.aws.amazon.com/AmazonS3/latest/API/API_DeleteBucketAnalyticsConfiguration.html) | ❌ Missing | ❌| ❌| ❌| ❌| | [DeleteBucketIntelligentTieringConfiguration](https://docs.aws.amazon.com/AmazonS3/latest/API/API_DeleteBucketIntelligentTieringConfiguration.html) | ❌ Missing | ❌| ❌| ❌| ❌| -- cgit v1.2.3 From 77e3fd6db2c9cd3a10889bd071e95ef839cfbefc Mon Sep 17 00:00:00 2001 From: Alex Date: Wed, 15 Jun 2022 20:20:28 +0200 Subject: improve internal item counter mechanisms and implement bucket quotas (#326) - [x] Refactoring of internal counting API - [x] Repair procedure for counters (it's an offline procedure!!!) - [x] New counter for objects in buckets - [x] Add quotas to buckets struct - [x] Add CLI to manage bucket quotas - [x] Add admin API to manage bucket quotas - [x] Apply quotas by adding checks on put operations - [x] Proof-read Co-authored-by: Alex Auvolat Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/326 Co-authored-by: Alex Co-committed-by: Alex --- doc/book/reference-manual/admin-api.md | 158 ++++++++++++++++++--------------- 1 file changed, 88 insertions(+), 70 deletions(-) (limited to 'doc/book/reference-manual') diff --git a/doc/book/reference-manual/admin-api.md b/doc/book/reference-manual/admin-api.md index b77f0d39..c7316cdf 100644 --- a/doc/book/reference-manual/admin-api.md +++ b/doc/book/reference-manual/admin-api.md @@ -134,8 +134,8 @@ Example request body: ```json [ - "ec79480e0ce52ae26fd00c9da684e4fa56658d9c64cdcecb094e936de0bfe71f@10.0.0.11:3901", - "4a6ae5a1d0d33bf895f5bb4f0a418b7dc94c47c0dd2eb108d1158f3c8f60b0ff@10.0.0.12:3901" + "ec79480e0ce52ae26fd00c9da684e4fa56658d9c64cdcecb094e936de0bfe71f@10.0.0.11:3901", + "4a6ae5a1d0d33bf895f5bb4f0a418b7dc94c47c0dd2eb108d1158f3c8f60b0ff@10.0.0.12:3901" ] ``` @@ -145,14 +145,14 @@ Example response: ```json [ - { - "success": true, - "error": null - }, - { - "success": false, - "error": "Handshake error" - } + { + "success": true, + "error": null + }, + { + "success": false, + "error": "Handshake error" + } ] ``` @@ -301,7 +301,7 @@ Request body format: ```json { - "name": "NameOfMyKey" + "name": "NameOfMyKey" } ``` @@ -313,9 +313,9 @@ Request body format: ```json { - "accessKeyId": "GK31c2f218a2e44f485b94239e", - "secretAccessKey": "b892c0665f0ada8a4755dae98baa3b133590e11dae3bcc1f9d769d67f16c3835", - "name": "NameOfMyKey" + "accessKeyId": "GK31c2f218a2e44f485b94239e", + "secretAccessKey": "b892c0665f0ada8a4755dae98baa3b133590e11dae3bcc1f9d769d67f16c3835", + "name": "NameOfMyKey" } ``` @@ -403,11 +403,11 @@ Request body format: ```json { - "name": "NameOfMyKey", - "allow": { - "createBucket": true, - }, - "deny": {} + "name": "NameOfMyKey", + "allow": { + "createBucket": true, + }, + "deny": {} } ``` @@ -473,24 +473,31 @@ Example response: ```json { - "id": "e6a14cd6a27f48684579ec6b381c078ab11697e6bc8513b72b2f5307e25fff9b", - "globalAliases": [ - "alex" - ], - "keys": [ - { - "accessKeyId": "GK31c2f218a2e44f485b94239e", - "name": "alex", - "permissions": { - "read": true, - "write": true, - "owner": true - }, - "bucketLocalAliases": [ - "test" - ] - } - ] + "id": "afa8f0a22b40b1247ccd0affb869b0af5cff980924a20e4b5e0720a44deb8d39", + "globalAliases": [], + "websiteAccess": false, + "websiteConfig": null, + "keys": [ + { + "accessKeyId": "GK31c2f218a2e44f485b94239e", + "name": "Imported key", + "permissions": { + "read": true, + "write": true, + "owner": true + }, + "bucketLocalAliases": [ + "debug" + ] + } + ], + "objects": 14827, + "bytes": 13189855625, + "unfinshedUploads": 0, + "quotas": { + "maxSize": null, + "maxObjects": null + } } ``` @@ -502,7 +509,7 @@ Request body format: ```json { - "globalAlias": "NameOfMyBucket" + "globalAlias": "NameOfMyBucket" } ``` @@ -510,15 +517,15 @@ OR ```json { - "localAlias": { - "accessKeyId": "GK31c2f218a2e44f485b94239e", - "alias": "NameOfMyBucket", - "allow": { - "read": true, - "write": true, - "owner": false - } - } + "localAlias": { + "accessKeyId": "GK31c2f218a2e44f485b94239e", + "alias": "NameOfMyBucket", + "allow": { + "read": true, + "write": true, + "owner": false + } + } } ``` @@ -540,26 +547,37 @@ Deletes a storage bucket. A bucket cannot be deleted if it is not empty. Warning: this will delete all aliases associated with the bucket! -#### PutBucketWebsite `PUT /v0/bucket/website?id=` +#### UpdateBucket `PUT /v0/bucket?id=` -Sets the website configuration for a bucket (this also enables website access for this bucket). +Updates configuration of the given bucket. Request body format: ```json { - "indexDocument": "index.html", - "errorDocument": "404.html" + "websiteAccess": { + "enabled": true, + "indexDocument": "index.html", + "errorDocument": "404.html" + }, + "quotas": { + "maxSize": 19029801, + "maxObjects": null, + } } ``` -The field `errorDocument` is optional, if no error document is set a generic error message is displayed when errors happen. - - -#### DeleteBucketWebsite `DELETE /v0/bucket/website?id=` +All fields (`websiteAccess` and `quotas`) are optionnal. +If they are present, the corresponding modifications are applied to the bucket, otherwise nothing is changed. -Deletes the website configuration for a bucket (disables website access for this bucket). +In `websiteAccess`: if `enabled` is `true`, `indexDocument` must be specified. +The field `errorDocument` is optional, if no error document is set a generic +error message is displayed when errors happen. Conversely, if `enabled` is +`false`, neither `indexDocument` nor `errorDocument` must be specified. +In `quotas`: new values of `maxSize` and `maxObjects` must both be specified, or set to `null` +to remove the quotas. An absent value will be considered the same as a `null`. It is not possible +to change only one of the two quotas. ### Operations on permissions for keys on buckets @@ -571,13 +589,13 @@ Request body format: ```json { - "bucketId": "e6a14cd6a27f48684579ec6b381c078ab11697e6bc8513b72b2f5307e25fff9b", - "accessKeyId": "GK31c2f218a2e44f485b94239e", - "permissions": { - "read": true, - "write": true, - "owner": true - }, + "bucketId": "e6a14cd6a27f48684579ec6b381c078ab11697e6bc8513b72b2f5307e25fff9b", + "accessKeyId": "GK31c2f218a2e44f485b94239e", + "permissions": { + "read": true, + "write": true, + "owner": true + }, } ``` @@ -592,13 +610,13 @@ Request body format: ```json { - "bucketId": "e6a14cd6a27f48684579ec6b381c078ab11697e6bc8513b72b2f5307e25fff9b", - "accessKeyId": "GK31c2f218a2e44f485b94239e", - "permissions": { - "read": false, - "write": false, - "owner": true - }, + "bucketId": "e6a14cd6a27f48684579ec6b381c078ab11697e6bc8513b72b2f5307e25fff9b", + "accessKeyId": "GK31c2f218a2e44f485b94239e", + "permissions": { + "read": false, + "write": false, + "owner": true + }, } ``` -- cgit v1.2.3 From b74b533b7be4f43bb565a5f025cf6121933307f6 Mon Sep 17 00:00:00 2001 From: Alex Date: Wed, 29 Jun 2022 11:50:51 +0200 Subject: Fix typo --- doc/book/reference-manual/s3-compatibility.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc/book/reference-manual') diff --git a/doc/book/reference-manual/s3-compatibility.md b/doc/book/reference-manual/s3-compatibility.md index a8e503d5..3d571264 100644 --- a/doc/book/reference-manual/s3-compatibility.md +++ b/doc/book/reference-manual/s3-compatibility.md @@ -148,7 +148,7 @@ Please open an issue if you have a use case for replication. | [PutBucketReplication](https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutBucketReplication.html) | ❌ Missing | ❌| ⚠ | ❌| ❌| *Note: Ceph documentation briefly says that Ceph supports -[replication though the S3 API](https://docs.ceph.com/en/latest/radosgw/multisite-sync-policy/#s3-replication-api) +[replication through the S3 API](https://docs.ceph.com/en/latest/radosgw/multisite-sync-policy/#s3-replication-api) but with some limitations. Additionaly, replication endpoints are not documented in the S3 compatibility page so I don't know what kind of support we can expect.* -- cgit v1.2.3 From f6aebefcc9747bf5afad3767e9ae6f9f3aba30ae Mon Sep 17 00:00:00 2001 From: Alex Auvolat Date: Wed, 14 Sep 2022 19:31:13 +0200 Subject: Some work on documentation towards v0.8 --- doc/book/reference-manual/admin-api.md | 2 +- doc/book/reference-manual/cli.md | 2 +- doc/book/reference-manual/configuration.md | 14 +---- doc/book/reference-manual/features.md | 85 +++++++++++++++++++++++++++ doc/book/reference-manual/k2v.md | 2 +- doc/book/reference-manual/layout.md | 2 +- doc/book/reference-manual/routing.md | 45 -------------- doc/book/reference-manual/s3-compatibility.md | 2 +- 8 files changed, 91 insertions(+), 63 deletions(-) create mode 100644 doc/book/reference-manual/features.md delete mode 100644 doc/book/reference-manual/routing.md (limited to 'doc/book/reference-manual') diff --git a/doc/book/reference-manual/admin-api.md b/doc/book/reference-manual/admin-api.md index c7316cdf..3a4a7aab 100644 --- a/doc/book/reference-manual/admin-api.md +++ b/doc/book/reference-manual/admin-api.md @@ -1,6 +1,6 @@ +++ title = "Administration API" -weight = 16 +weight = 60 +++ The Garage administration API is accessible through a dedicated server whose diff --git a/doc/book/reference-manual/cli.md b/doc/book/reference-manual/cli.md index 43a0c823..82492c3e 100644 --- a/doc/book/reference-manual/cli.md +++ b/doc/book/reference-manual/cli.md @@ -1,6 +1,6 @@ +++ title = "Garage CLI" -weight = 15 +weight = 30 +++ The Garage CLI is mostly self-documented. Make use of the `help` subcommand diff --git a/doc/book/reference-manual/configuration.md b/doc/book/reference-manual/configuration.md index 65381f46..6db12568 100644 --- a/doc/book/reference-manual/configuration.md +++ b/doc/book/reference-manual/configuration.md @@ -1,6 +1,6 @@ +++ title = "Configuration file format" -weight = 5 +weight = 20 +++ Here is an example `garage.toml` configuration file that illustrates all of the possible options: @@ -10,7 +10,6 @@ metadata_dir = "/var/lib/garage/meta" data_dir = "/var/lib/garage/data" block_size = 1048576 -block_manager_background_tranquility = 2 replication_mode = "3" @@ -87,17 +86,6 @@ files will remain available. This however means that chunks from existing files will not be deduplicated with chunks from newly uploaded files, meaning you might use more storage space that is optimally possible. -### `block_manager_background_tranquility` - -This parameter tunes the activity of the background worker responsible for -resyncing data blocks between nodes. The higher the tranquility value is set, -the more the background worker will wait between iterations, meaning the load -on the system (including network usage between nodes) will be reduced. The -minimal value for this parameter is `0`, where the background worker will -allways work at maximal throughput to resynchronize blocks. The default value -is `2`, where the background worker will try to spend at most 1/3 of its time -working, and 2/3 sleeping in order to reduce system load. - ### `replication_mode` Garage supports the following replication modes: diff --git a/doc/book/reference-manual/features.md b/doc/book/reference-manual/features.md new file mode 100644 index 00000000..23750800 --- /dev/null +++ b/doc/book/reference-manual/features.md @@ -0,0 +1,85 @@ ++++ +title = "List of Garage features" +weight = 10 ++++ + + +### S3 API + +The main goal of Garage is to provide an object storage service that is compatible with the +[S3 API](https://docs.aws.amazon.com/AmazonS3/latest/API/Welcome.html) from Amazon Web Services. +We try to adhere as strictly as possible to the semantics of the API as implemented by Amazon +and other vendors such as Minio or CEPH. + +Of course Garage does not implement the full span of API endpoints that AWS S3 does; +the exact list of S3 features implemented by Garage can be found [on our S3 compatibility page](@/documentation/reference-manual/s3-compatibility.md). + +### Geo-distribution + +Garage allows you to store copies of your data in multiple geographical locations in order to maximize resilience +to adverse events, such as network/power outages or hardware failures. +This allows Garage to run very well even at home, using consumer-grade Internet connectivity +(such as FTTH) and power, as long as cluster nodes can be spawned at several physical locations. +Garage exploits knowledge of the capacity and physical location of each storage node to design +a storage plan that best exploits the available storage capacity while satisfying the geo-distributed replication constraint. + +To learn more about geo-distributed Garage clusters, +read our documentation on [setting up a real-world deployment](@/documentation/cookbook/real-world.md). + +### Flexible topology + +A Garage cluster can very easily evolve over time, as storage nodes are added or removed. +Garage will automatically rebalance data between nodes as needed to ensure the desired number of copies. +Read about cluster layout management [here](@/documentation/reference-manual/layout.md). + +### No RAFT slowing you down + +It might seem strange to tout the absence of something as a desirable feature, +but this is in fact a very important point! Garage does not use RAFT or another +consensus algorithm internally to order incoming requests: this means that all requests +directed to a Garage cluster can be handled independently of one another instead +of going through a central bottleneck (the leader node). +As a consequence, requests can be handled much faster, even in cases where latency +between cluster nodes is important (see our [benchmarks](@/documentation/design/benchmarks/index.md) for data on this). +This is particularly usefull when nodes are far from one another and talk to one other through standard Internet connections. + +### Several replication modes + +Garage supports a variety of replication modes, with 1 copy, 2 copies or 3 copies of your data, +and with various levels of consistency. +Read our reference page on [supported replication modes](@/documentation/reference-manual/configuration.md#replication-mode) +to select the replication mode best suited to your use case (hint: in most cases, `replication_mode = "3"` is what you want). + +### Web server for static websites + +A storage bucket can easily be configured to be served directly by Garage as a static web site. +Domain names for multiple websites directly map to bucket names, making it easy to build +a platform for your user's to autonomously build and host their websites over Garage. +Surprisingly, none of the other alternative S3 implementations we surveyed (such as Minio +or CEPH) support publishing static websites from S3 buckets, a feature that is however +directly inherited from S3 on AWS. + +### Bucket names as aliases + + - the same bucket may have multiple names (useful when exposing websites for example) + + - bucket renaming is possible + + - Scoped buckets: 2 users can have a different bucket with the same name -> avoid collision. Helpful if you want to write an application that creates per-user bucket always with the same name. + +### Standalone/self contained + + +### Integration with Kubernetes and Nomad + +Many node discovery methods: Kubernetes integration, Nomad integration through Consul + +### Support for changing IP addresses + +(as long as all nodes don't change their IP at the same time) + +### Cluster administration API + +### Metrics and traces + +### (experimental) K2V API diff --git a/doc/book/reference-manual/k2v.md b/doc/book/reference-manual/k2v.md index 742e4309..207d056a 100644 --- a/doc/book/reference-manual/k2v.md +++ b/doc/book/reference-manual/k2v.md @@ -1,6 +1,6 @@ +++ title = "K2V" -weight = 30 +weight = 70 +++ Starting with version 0.7.2, Garage introduces an optionnal feature, K2V, diff --git a/doc/book/reference-manual/layout.md b/doc/book/reference-manual/layout.md index 7debbf33..a7d6f51f 100644 --- a/doc/book/reference-manual/layout.md +++ b/doc/book/reference-manual/layout.md @@ -1,6 +1,6 @@ +++ title = "Cluster layout management" -weight = 10 +weight = 50 +++ The cluster layout in Garage is a table that assigns to each node a role in diff --git a/doc/book/reference-manual/routing.md b/doc/book/reference-manual/routing.md deleted file mode 100644 index aec637cc..00000000 --- a/doc/book/reference-manual/routing.md +++ /dev/null @@ -1,45 +0,0 @@ -+++ -title = "Request routing logic" -weight = 10 -+++ - -Data retrieval requests to Garage endpoints (S3 API and websites) are resolved -to an individual object in a bucket. Since objects are replicated to multiple nodes -Garage must ensure consistency before answering the request. - -## Using quorum to ensure consistency - -Garage ensures consistency by attempting to establish a quorum with the -data nodes responsible for the object. When a majority of the data nodes -have provided metadata on a object Garage can then answer the request. - -When a request arrives Garage will, assuming the recommended 3 replicas, perform the following actions: - -- Make a request to the two preferred nodes for object metadata -- Try the third node if one of the two initial requests fail -- Check that the metadata from at least 2 nodes match -- Check that the object hasn't been marked deleted -- Answer the request with inline data from metadata if object is small enough -- Or get data blocks from the preferred nodes and answer using the assembled object - -Garage dynamically determines which nodes to query based on health, preference, and -which nodes actually host a given data. Garage has no concept of "primary" so any -healthy node with the data can be used as long as a quorum is reached for the metadata. - -## Node health - -Garage keeps a TCP session open to each node in the cluster and periodically pings them. If a connection -cannot be established, or a node fails to answer a number of pings, the target node is marked as failed. -Failed nodes are not used for quorum or other internal requests. - -## Node preference - -Garage prioritizes which nodes to query according to a few criteria: - -- A node always prefers itself if it can answer the request -- Then the node prioritizes nodes in the same zone -- Finally the nodes with the lowest latency are prioritized - - -For further reading on the cluster structure look at the [gateway](@/documentation/cookbook/gateways.md) -and [cluster layout management](@/documentation/reference-manual/layout.md) pages. \ No newline at end of file diff --git a/doc/book/reference-manual/s3-compatibility.md b/doc/book/reference-manual/s3-compatibility.md index 3d571264..dd3492a0 100644 --- a/doc/book/reference-manual/s3-compatibility.md +++ b/doc/book/reference-manual/s3-compatibility.md @@ -1,6 +1,6 @@ +++ title = "S3 Compatibility status" -weight = 20 +weight = 40 +++ ## DISCLAIMER -- cgit v1.2.3 From 1d0a610690dbc711bf22d751ea3e6fe7047dc0a4 Mon Sep 17 00:00:00 2001 From: Alex Auvolat Date: Thu, 15 Sep 2022 13:23:57 +0200 Subject: Finish writing about Garage features, and fix from-source instructions --- doc/book/reference-manual/features.md | 72 +++++++++++++++++++++++++++-------- 1 file changed, 56 insertions(+), 16 deletions(-) (limited to 'doc/book/reference-manual') diff --git a/doc/book/reference-manual/features.md b/doc/book/reference-manual/features.md index 23750800..d2d28946 100644 --- a/doc/book/reference-manual/features.md +++ b/doc/book/reference-manual/features.md @@ -26,6 +26,11 @@ a storage plan that best exploits the available storage capacity while satisfyin To learn more about geo-distributed Garage clusters, read our documentation on [setting up a real-world deployment](@/documentation/cookbook/real-world.md). +### Standalone/self-contained + +Garage is extremely simple to deploy, and does not depend on any external service to run. +This makes setting up and administering storage clusters, we hope, as easy as it could be. + ### Flexible topology A Garage cluster can very easily evolve over time, as storage nodes are added or removed. @@ -42,11 +47,11 @@ of going through a central bottleneck (the leader node). As a consequence, requests can be handled much faster, even in cases where latency between cluster nodes is important (see our [benchmarks](@/documentation/design/benchmarks/index.md) for data on this). This is particularly usefull when nodes are far from one another and talk to one other through standard Internet connections. - + ### Several replication modes Garage supports a variety of replication modes, with 1 copy, 2 copies or 3 copies of your data, -and with various levels of consistency. +and with various levels of consistency, in order to adapt to a variety of usage scenarios. Read our reference page on [supported replication modes](@/documentation/reference-manual/configuration.md#replication-mode) to select the replication mode best suited to your use case (hint: in most cases, `replication_mode = "3"` is what you want). @@ -54,32 +59,67 @@ to select the replication mode best suited to your use case (hint: in most cases A storage bucket can easily be configured to be served directly by Garage as a static web site. Domain names for multiple websites directly map to bucket names, making it easy to build -a platform for your user's to autonomously build and host their websites over Garage. +a platform for your users to autonomously build and host their websites over Garage. Surprisingly, none of the other alternative S3 implementations we surveyed (such as Minio or CEPH) support publishing static websites from S3 buckets, a feature that is however directly inherited from S3 on AWS. +Read more on our [dedicated documentation page](@/documentation/cookbook/exposing-websites.md). ### Bucket names as aliases - - the same bucket may have multiple names (useful when exposing websites for example) +In Garage, a bucket may have several names, known as aliases. +Aliases can easily be added and removed on demand: +this allows to easily rename buckets if needed +without having to copy all of their content, something that cannot be done on AWS. +For buckets served as static websites, having multiple aliases for a bucket can allow +exposing the same content under different domain names. + +Garage also supports bucket aliases which are local to a single user: +this allows different users to have different buckets with the same name, thus avoiding naming collisions. +This can be helpfull for instance if you want to write an application that creates per-user buckets with always the same name. + +This feature is totally invisible to S3 clients and does not break compatibility with AWS. + +### Cluster administration API + +Garage provides a fully-fledged REST API to administer your cluster programatically. +Functionnality included in the admin API include: setting up and monitoring +cluster nodes, managing access credentials, and managing storage buckets and bucket aliases. +A full reference of the administration API is available [here](@/documentation/reference-manual/admin-api.md). + +### Metrics and traces - - bucket renaming is possible +Garage makes some internal metrics available in the Prometheus data format, +which allows you to build interactive dashboards to visualize the load and internal state of your storage cluster. - - Scoped buckets: 2 users can have a different bucket with the same name -> avoid collision. Helpful if you want to write an application that creates per-user bucket always with the same name. +For developpers and performance-savvy administrators, +Garage also supports exporting traces of what it does internally in OpenTelemetry format. +This allows to monitor the time spent at various steps of the processing of requests, +in order to detect potential performance bottlenecks. -### Standalone/self contained +### Kubernetes and Nomad integrations - -### Integration with Kubernetes and Nomad +Garage can automatically discover other nodes in the cluster thanks to integration +with orchestrators such as Kubernetes and Nomad (when used with Consul). +This eases the configuration of your cluster as it removes one step where nodes need +to be manually connected to one another. -Many node discovery methods: Kubernetes integration, Nomad integration through Consul - ### Support for changing IP addresses -(as long as all nodes don't change their IP at the same time) +As long as all of your nodes don't thange their IP address at the same time, +Garage should be able to tolerate nodes with changing/dynamic IP addresses, +as nodes will regularly exchange the IP addresses of their peers and try to +reconnect using newer addresses when existing connections are broken. -### Cluster administration API +### K2V API (experimental) -### Metrics and traces - -### (experimental) K2V API +As part of an ongoing research project, Garage can expose an experimental key/value storage API called K2V. +K2V is made for the storage and retrieval of many small key/value pairs that need to be processed in bulk. +This completes the S3 API with an alternative that can be used to easily store and access metadata +related to objects stored in an S3 bucket. + +In the context of our research project, [Aérogramme](https://aerogramme.deuxfleurs.fr), +K2V is used to provide metadata and log storage for operations on encrypted e-mail storage. + +Learn more on the specification of K2V [here](https://git.deuxfleurs.fr/Deuxfleurs/garage/src/branch/k2v/doc/drafts/k2v-spec.md) +and on how to enable it in Garage [here](@/documentation/reference-manual/k2v.md). -- cgit v1.2.3 From 0ab0d3cc29d0d3779762be6d036317f7608a394c Mon Sep 17 00:00:00 2001 From: Alex Auvolat Date: Tue, 27 Sep 2022 16:52:36 +0200 Subject: Document db_engine --- doc/book/reference-manual/configuration.md | 43 ++++++++++++++++++++++++++++++ 1 file changed, 43 insertions(+) (limited to 'doc/book/reference-manual') diff --git a/doc/book/reference-manual/configuration.md b/doc/book/reference-manual/configuration.md index 6db12568..97da0e0e 100644 --- a/doc/book/reference-manual/configuration.md +++ b/doc/book/reference-manual/configuration.md @@ -9,6 +9,8 @@ Here is an example `garage.toml` configuration file that illustrates all of the metadata_dir = "/var/lib/garage/meta" data_dir = "/var/lib/garage/data" +db_engine = "lmdb" + block_size = 1048576 replication_mode = "3" @@ -71,6 +73,47 @@ This folder can be placed on an HDD. The space available for `data_dir` should be counted to determine a node's capacity when [adding it to the cluster layout](@/documentation/cookbook/real-world.md). +### `db_engine` (since `v0.8.0`) + +By default, Garage uses the Sled embedded database library +to store its metadata on-disk. Since `v0.8.0`, Garage can use alternative storage backends as follows: + +| DB engine | `db_engine` value | Database path | +| --------- | ----------------- | ------------- | +| [Sled](https://sled.rs) | `"sled"` | `/db/` | +| [LMDB](https://www.lmdb.tech) | `"lmdb"` | `/db.lmdb/` | +| [Sqlite](https://sqlite.org) | `"sqlite"` | `/db.sqlite` | + +Performance characteristics of the different DB engines are as follows: + +- Sled: the default database engine, which tends to produce + large data files and also has performance issues, especially when the metadata folder + is on a traditionnal HDD and not on SSD. +- LMDB: the recommended alternative on 64-bit systems, + much more space-efficiant and slightly faster. Note that the data format of LMDB is not portable + between architectures, so for instance the Garage database of an x86-64 + node cannot be moved to an ARM64 node. Also note that, while LMDB can technically be used on 32-bit systems, + this will limit your node to very small database sizes due to how LMDB works; it is therefore not recommended. +- Sqlite: Garage supports Sqlite as a storage backend for metadata, + however it may have issues and is also very slow in its current implementation, + so it is not recommended to be used for now. + +It is possible to convert Garage's metadata directory from one format to another with a small utility named `convert_db`, +which can be downloaded at the following locations: +[for amd64](https://garagehq.deuxfleurs.fr/_releases/convert_db/amd64/convert_db), +[for i386](https://garagehq.deuxfleurs.fr/_releases/convert_db/i386/convert_db), +[for arm64](https://garagehq.deuxfleurs.fr/_releases/convert_db/arm64/convert_db), +[for arm](https://garagehq.deuxfleurs.fr/_releases/convert_db/arm/convert_db). +The `convert_db` utility is used as folows: + +``` +convert-db -a -i \ + -b -o +``` + +Make sure to specify the full database path as presented in the table above, +and not just the path to the metadata directory. + ### `block_size` Garage splits stored objects in consecutive chunks of size `block_size` -- cgit v1.2.3