diff options
Diffstat (limited to 'doc/book/reference-manual')
| -rw-r--r-- | doc/book/reference-manual/_index.md | 2 | ||||
| -rw-r--r-- | doc/book/reference-manual/admin-api.md | 90 | ||||
| -rw-r--r-- | doc/book/reference-manual/configuration.md | 44 | ||||
| -rw-r--r-- | doc/book/reference-manual/features.md | 2 | ||||
| -rw-r--r-- | doc/book/reference-manual/k2v.md | 2 | ||||
| -rw-r--r-- | doc/book/reference-manual/layout.md | 77 |
6 files changed, 130 insertions, 87 deletions
diff --git a/doc/book/reference-manual/_index.md b/doc/book/reference-manual/_index.md index ab1de5e6..1f360e57 100644 --- a/doc/book/reference-manual/_index.md +++ b/doc/book/reference-manual/_index.md @@ -1,6 +1,6 @@ +++ title = "Reference Manual" -weight = 5 +weight = 60 sort_by = "weight" template = "documentation.html" +++ diff --git a/doc/book/reference-manual/admin-api.md b/doc/book/reference-manual/admin-api.md index 363bc886..6932ac60 100644 --- a/doc/book/reference-manual/admin-api.md +++ b/doc/book/reference-manual/admin-api.md @@ -39,11 +39,95 @@ Authorization: Bearer <token> ## Administration API endpoints -### Metrics-related endpoints - -#### Metrics `GET /metrics` +### Metrics `GET /metrics` Returns internal Garage metrics in Prometheus format. +The metrics are directly documented when returned by the API. + +**Example:** + +``` +$ curl -i http://localhost:3903/metrics +HTTP/1.1 200 OK +content-type: text/plain; version=0.0.4 +content-length: 12145 +date: Tue, 08 Aug 2023 07:25:05 GMT + +# HELP api_admin_error_counter Number of API calls to the various Admin API endpoints that resulted in errors +# TYPE api_admin_error_counter counter +api_admin_error_counter{api_endpoint="CheckWebsiteEnabled",status_code="400"} 1 +api_admin_error_counter{api_endpoint="CheckWebsiteEnabled",status_code="404"} 3 +# HELP api_admin_request_counter Number of API calls to the various Admin API endpoints +# TYPE api_admin_request_counter counter +api_admin_request_counter{api_endpoint="CheckWebsiteEnabled"} 7 +api_admin_request_counter{api_endpoint="Health"} 3 +# HELP api_admin_request_duration Duration of API calls to the various Admin API endpoints +... +``` + +### Health `GET /health` + +Returns `200 OK` if enough nodes are up to have a quorum (ie. serve requests), +otherwise returns `503 Service Unavailable`. + +**Example:** + +``` +$ curl -i http://localhost:3903/health +HTTP/1.1 200 OK +content-type: text/plain +content-length: 102 +date: Tue, 08 Aug 2023 07:22:38 GMT + +Garage is fully operational +Consult the full health check API endpoint at /v0/health for more details +``` + +### On-demand TLS `GET /check` + +To prevent abuses for on-demand TLS, Caddy developpers have specified an endpoint that can be queried by the reverse proxy +to know if a given domain is allowed to get a certificate. Garage implements this endpoints to tell if a given domain is handled by Garage or is garbage. + +Garage responds with the following logic: + - If the domain matches the pattern `<bucket-name>.<s3_api.root_domain>`, returns 200 OK + - If the domain matches the pattern `<bucket-name>.<s3_web.root_domain>` and website is configured for `<bucket>`, returns 200 OK + - If the domain matches the pattern `<bucket-name>` and website is configured for `<bucket>`, returns 200 OK + - Otherwise, returns 404 Not Found, 400 Bad Request or 5xx requests. + +*Note 1: because in the path-style URL mode, there is only one domain that is not known by Garage, hence it is not supported by this API endpoint. +You must manually declare the domain in your reverse-proxy. Idem for K2V.* + +*Note 2: buckets in a user's namespace are not supported yet by this endpoint. This is a limitation of this endpoint currently.* + +**Example:** Suppose a Garage instance configured with `s3_api.root_domain = .s3.garage.localhost` and `s3_web.root_domain = .web.garage.localhost`. + +With a private `media` bucket (name in the global namespace, website is disabled), the endpoint will feature the following behavior: + +``` +$ curl -so /dev/null -w "%{http_code}" http://localhost:3903/check?domain=media.s3.garage.localhost +200 +$ curl -so /dev/null -w "%{http_code}" http://localhost:3903/check?domain=media +400 +$ curl -so /dev/null -w "%{http_code}" http://localhost:3903/check?domain=media.web.garage.localhost +400 +``` + +With a public `example.com` bucket (name in the global namespace, website is activated), the endpoint will feature the following behavior: + +``` +$ curl -so /dev/null -w "%{http_code}" http://localhost:3903/check?domain=example.com.s3.garage.localhost +200 +$ curl -so /dev/null -w "%{http_code}" http://localhost:3903/check?domain=example.com +200 +$ curl -so /dev/null -w "%{http_code}" http://localhost:3903/check?domain=example.com.web.garage.localhost +200 +``` + + +**References:** + - [Using On-Demand TLS](https://caddyserver.com/docs/automatic-https#using-on-demand-tls) + - [Add option for a backend check to approve use of on-demand TLS](https://github.com/caddyserver/caddy/pull/1939) + - [Serving tens of thousands of domains over HTTPS with Caddy](https://caddy.community/t/serving-tens-of-thousands-of-domains-over-https-with-caddy/11179) ### Cluster operations diff --git a/doc/book/reference-manual/configuration.md b/doc/book/reference-manual/configuration.md index 38062bab..b916bb61 100644 --- a/doc/book/reference-manual/configuration.md +++ b/doc/book/reference-manual/configuration.md @@ -35,12 +35,18 @@ bootstrap_peers = [ [consul_discovery] +api = "catalog" consul_http_addr = "http://127.0.0.1:8500" service_name = "garage-daemon" ca_cert = "/etc/consul/consul-ca.crt" client_cert = "/etc/consul/consul-client.crt" client_key = "/etc/consul/consul-key.crt" +# for `agent` API mode, unset client_cert and client_key, and optionally enable `token` +# token = "abcdef-01234-56789" tls_skip_verify = false +tags = [ "dns-enabled" ] +meta = { dns-acl = "allow trusted" } + [kubernetes_discovery] namespace = "garage" @@ -201,7 +207,7 @@ Garage supports the following replication modes: that should probably never be used. Note that in modes `2` and `3`, -if at least the same number of zones are available, an arbitrary number of failures in +if at least the same number of zones are available, an arbitrary number of failures in any given zone is tolerated as copies of data will be spread over several zones. **Make sure `replication_mode` is the same in the configuration files of all nodes. @@ -245,7 +251,7 @@ Values between `1` (faster compression) and `19` (smaller file) are standard com levels for zstd. From `20` to `22`, compression levels are referred as "ultra" and must be used with extra care as it will use lot of memory. A value of `0` will let zstd choose a default value (currently `3`). Finally, zstd has also compression designed to be faster -than default compression levels, they range from `-1` (smaller file) to `-99` (faster +than default compression levels, they range from `-1` (smaller file) to `-99` (faster compression). If you do not specify a `compression_level` entry, Garage will set it to `1` for you. With @@ -316,6 +322,12 @@ reached by other nodes of the cluster, which should be set in `rpc_public_addr`. The `consul_http_addr` parameter should be set to the full HTTP(S) address of the Consul server. +### `api` + +Two APIs for service registration are supported: `catalog` and `agent`. `catalog`, the default, will register a service using +the `/v1/catalog` endpoints, enabling mTLS if `client_cert` and `client_key` are provided. The `agent` API uses the +`v1/agent` endpoints instead, where an optional `token` may be provided. + ### `service_name` `service_name` should be set to the service name under which Garage's @@ -324,6 +336,7 @@ RPC ports are announced. ### `client_cert`, `client_key` TLS client certificate and client key to use when communicating with Consul over TLS. Both are mandatory when doing so. +Only available when `api = "catalog"`. ### `ca_cert` @@ -334,6 +347,29 @@ TLS CA certificate to use when communicating with Consul over TLS. Skip server hostname verification in TLS handshake. `ca_cert` is ignored when this is set. +### `token` + +Uses the provided token for communication with Consul. Only available when `api = "agent"`. +The policy assigned to this token should at least have these rules: + +```hcl +// the `service_name` specified above +service "garage" { + policy = "write" +} + +service_prefix "" { + policy = "read" +} + +node_prefix "" { + policy = "read" +} +``` + +### `tags` and `meta` + +Additional list of tags and map of service meta to add during service registration. ## The `[kubernetes_discovery]` section @@ -373,7 +409,7 @@ message that redirects the client to the correct region. ### `root_domain` {#root_domain} -The optionnal suffix to access bucket using vhost-style in addition to path-style request. +The optional suffix to access bucket using vhost-style in addition to path-style request. Note path-style requests are always enabled, whether or not vhost-style is configured. Configuring vhost-style S3 required a wildcard DNS entry, and possibly a wildcard TLS certificate, but might be required by softwares not supporting path-style requests. @@ -396,7 +432,7 @@ This endpoint does not suport TLS: a reverse proxy should be used to provide it. ### `root_domain` -The optionnal suffix appended to bucket names for the corresponding HTTP Host. +The optional suffix appended to bucket names for the corresponding HTTP Host. For instance, if `root_domain` is `web.garage.eu`, a bucket called `deuxfleurs.fr` will be accessible either with hostname `deuxfleurs.fr.web.garage.eu` diff --git a/doc/book/reference-manual/features.md b/doc/book/reference-manual/features.md index 550504ff..2f8e633a 100644 --- a/doc/book/reference-manual/features.md +++ b/doc/book/reference-manual/features.md @@ -35,7 +35,7 @@ This makes setting up and administering storage clusters, we hope, as easy as it A Garage cluster can very easily evolve over time, as storage nodes are added or removed. Garage will automatically rebalance data between nodes as needed to ensure the desired number of copies. -Read about cluster layout management [here](@/documentation/reference-manual/layout.md). +Read about cluster layout management [here](@/documentation/operations/layout.md). ### No RAFT slowing you down diff --git a/doc/book/reference-manual/k2v.md b/doc/book/reference-manual/k2v.md index ed069b27..c01f641e 100644 --- a/doc/book/reference-manual/k2v.md +++ b/doc/book/reference-manual/k2v.md @@ -3,7 +3,7 @@ title = "K2V" weight = 100 +++ -Starting with version 0.7.2, Garage introduces an optionnal feature, K2V, +Starting with version 0.7.2, Garage introduces an optional feature, K2V, which is an alternative storage API designed to help efficiently store many small values in buckets (in opposition to S3 which is more designed to store large blobs). diff --git a/doc/book/reference-manual/layout.md b/doc/book/reference-manual/layout.md deleted file mode 100644 index a7d6f51f..00000000 --- a/doc/book/reference-manual/layout.md +++ /dev/null @@ -1,77 +0,0 @@ -+++ -title = "Cluster layout management" -weight = 50 -+++ - -The cluster layout in Garage is a table that assigns to each node a role in -the cluster. The role of a node in Garage can either be a storage node with -a certain capacity, or a gateway node that does not store data and is only -used as an API entry point for faster cluster access. -An introduction to building cluster layouts can be found in the [production deployment](@/documentation/cookbook/real-world.md) page. - -## How cluster layouts work in Garage - -In Garage, a cluster layout is composed of the following components: - -- a table of roles assigned to nodes -- a version number - -Garage nodes will always use the cluster layout with the highest version number. - -Garage nodes also maintain and synchronize between them a set of proposed role -changes that haven't yet been applied. These changes will be applied (or -canceled) in the next version of the layout - -The following commands insert modifications to the set of proposed role changes -for the next layout version (but they do not create the new layout immediately): - -```bash -garage layout assign [...] -garage layout remove [...] -``` - -The following command can be used to inspect the layout that is currently set in the cluster -and the changes proposed for the next layout version, if any: - -```bash -garage layout show -``` - -The following commands create a new layout with the specified version number, -that either takes into account the proposed changes or cancels them: - -```bash -garage layout apply --version <new_version_number> -garage layout revert --version <new_version_number> -``` - -The version number of the new layout to create must be 1 + the version number -of the previous layout that existed in the cluster. The `apply` and `revert` -commands will fail otherwise. - -## Warnings about Garage cluster layout management - -**Warning: never make several calls to `garage layout apply` or `garage layout -revert` with the same value of the `--version` flag. Doing so can lead to the -creation of several different layouts with the same version number, in which -case your Garage cluster will become inconsistent until fixed.** If a call to -`garage layout apply` or `garage layout revert` has failed and `garage layout -show` indicates that a new layout with the given version number has not been -set in the cluster, then it is fine to call the command again with the same -version number. - -If you are using the `garage` CLI by typing individual commands in your -shell, you shouldn't have much issues as long as you run commands one after -the other and take care of checking the output of `garage layout show` -before applying any changes. - -If you are using the `garage` CLI to script layout changes, follow the following recommendations: - -- Make all of your `garage` CLI calls to the same RPC host. Do not use the - `garage` CLI to connect to individual nodes to send them each a piece of the - layout changes you are making, as the changes propagate asynchronously - between nodes and might not all be taken into account at the time when the - new layout is applied. - -- **Only call `garage layout apply` once**, and call it **strictly after** all - of the `layout assign` and `layout remove` commands have returned. |