aboutsummaryrefslogtreecommitdiff
path: root/doc/book/cookbook
diff options
context:
space:
mode:
authorMendes <mendes.oulamara@pm.me>2022-10-04 18:14:49 +0200
committerMendes <mendes.oulamara@pm.me>2022-10-04 18:14:49 +0200
commit829f815a897b04986559910bbcbf53625adcdf20 (patch)
tree6db3c27cff2aded754a641d1f2b05c83be701267 /doc/book/cookbook
parent99f96b9564c9c841dc6c56f1255a6e70ff884d46 (diff)
parenta096ced35562bd0a8877a1ee2f755be1edafe343 (diff)
downloadgarage-829f815a897b04986559910bbcbf53625adcdf20.tar.gz
garage-829f815a897b04986559910bbcbf53625adcdf20.zip
Merge remote-tracking branch 'origin/main' into optimal-layout
Diffstat (limited to 'doc/book/cookbook')
-rw-r--r--doc/book/cookbook/exposing-websites.md4
-rw-r--r--doc/book/cookbook/from-source.md74
-rw-r--r--doc/book/cookbook/kubernetes.md87
-rw-r--r--doc/book/cookbook/real-world.md12
-rw-r--r--doc/book/cookbook/reverse-proxy.md165
5 files changed, 313 insertions, 29 deletions
diff --git a/doc/book/cookbook/exposing-websites.md b/doc/book/cookbook/exposing-websites.md
index be462dc9..5f6a5a28 100644
--- a/doc/book/cookbook/exposing-websites.md
+++ b/doc/book/cookbook/exposing-websites.md
@@ -5,12 +5,14 @@ weight = 25
## Configuring a bucket for website access
-There are two methods to expose buckets as website:
+There are three methods to expose buckets as website:
1. using the PutBucketWebsite S3 API call, which is allowed for access keys that have the owner permission bit set
2. from the Garage CLI, by an adminstrator of the cluster
+3. using the Garage administration API
+
The `PutBucketWebsite` API endpoint [is documented](https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutBucketWebsite.html) in the official AWS docs.
This endpoint can also be called [using `aws s3api`](https://docs.aws.amazon.com/cli/latest/reference/s3api/put-bucket-website.html) on the command line.
The website configuration supported by Garage is only a subset of the possibilities on Amazon S3: redirections are not supported, only the index document and error document can be specified.
diff --git a/doc/book/cookbook/from-source.md b/doc/book/cookbook/from-source.md
index 5973d411..bacf93ab 100644
--- a/doc/book/cookbook/from-source.md
+++ b/doc/book/cookbook/from-source.md
@@ -20,40 +20,76 @@ sudo apt-get update
sudo apt-get install build-essential
```
-## Using source from `crates.io`
+## Building from source from the Gitea repository
-Garage's source code is published on `crates.io`, Rust's official package repository.
-This means you can simply ask `cargo` to download and build this source code for you:
+The primary location for Garage's source code is the
+[Gitea repository](https://git.deuxfleurs.fr/Deuxfleurs/garage),
+which contains all of the released versions as well as the code
+for the developpement of the next version.
+
+Clone the repository and enter it as follows:
```bash
-cargo install garage
+git clone https://git.deuxfleurs.fr/Deuxfleurs/garage.git
+cd garage
```
-That's all, `garage` should be in `$HOME/.cargo/bin`.
-
-You can add this folder to your `$PATH` or copy the binary somewhere else on your system.
-For instance:
+If you wish to build a specific version of Garage, check out the corresponding tag. For instance:
```bash
-sudo cp $HOME/.cargo/bin/garage /usr/local/bin/garage
+git tag # List available tags
+git checkout v0.8.0 # Change v0.8.0 with the version you wish to build
```
+Otherwise you will be building a developpement build from the `main` branch
+that includes all of the changes to be released in the next version.
+Be careful that such a build might be unstable or contain bugs,
+and could be incompatible with nodes that run stable versions of Garage.
-## Using source from the Gitea repository
+Finally, build Garage with the following command:
-The primary location for Garage's source code is the
-[Gitea repository](https://git.deuxfleurs.fr/Deuxfleurs/garage).
+```bash
+cargo build --release
+```
-Clone the repository and build Garage with the following commands:
+The binary built this way can now be found in `target/release/garage`.
+You may simply copy this binary to somewhere in your `$PATH` in order to
+have the `garage` command available in your shell, for instance:
```bash
-git clone https://git.deuxfleurs.fr/Deuxfleurs/garage.git
-cd garage
-cargo build
+sudo cp target/release/garage /usr/local/bin/garage
```
-Be careful, as this will make a debug build of Garage, which will be extremely slow!
-To make a release build, invoke `cargo build --release` (this takes much longer).
+If you are planning to develop Garage,
+you might be interested in producing debug builds, which compile faster but run slower:
+this can be done by removing the `--release` flag, and the resulting build can then
+be found in `target/debug/garage`.
+
+## List of available Cargo feature flags
-The binaries built this way are found in `target/{debug,release}/garage`.
+Garage supports a number of compilation options in the form of Cargo feature flags,
+which can be used to provide builds adapted to your system and your use case.
+To produce a build with a given set of features, invoke the `cargo build` command
+as follows:
+
+```bash
+# This will build the default feature set plus feature1, feature2 and feature3
+cargo build --release --features feature1,feature2,feature3
+# This will build ONLY feature1, feature2 and feature3
+cargo build --release --no-default-features \
+ --features feature1,feature2,feature3
+```
+The following feature flags are available in v0.8.0:
+
+| Feature flag | Enabled | Description |
+| ------------ | ------- | ----------- |
+| `bundled-libs` | *by default* | Use bundled version of sqlite3, zstd, lmdb and libsodium |
+| `system-libs` | optional | Use system version of sqlite3, zstd, lmdb and libsodium<br>if available (exclusive with `bundled-libs`, build using<br>`cargo build --no-default-features --features system-libs`) |
+| `k2v` | optional | Enable the experimental K2V API (if used, all nodes on your<br>Garage cluster must have it enabled as well) |
+| `kubernetes-discovery` | optional | Enable automatic registration and discovery<br>of cluster nodes through the Kubernetes API |
+| `metrics` | *by default* | Enable collection of metrics in Prometheus format on the admin API |
+| `telemetry-otlp` | optional | Enable collection of execution traces using OpenTelemetry |
+| `sled` | *by default* | Enable using Sled to store Garage's metadata |
+| `lmdb` | optional | Enable using LMDB to store Garage's metadata |
+| `sqlite` | optional | Enable using Sqlite3 to store Garage's metadata |
diff --git a/doc/book/cookbook/kubernetes.md b/doc/book/cookbook/kubernetes.md
new file mode 100644
index 00000000..9eafe3e1
--- /dev/null
+++ b/doc/book/cookbook/kubernetes.md
@@ -0,0 +1,87 @@
++++
+title = "Deploying on Kubernetes"
+weight = 32
++++
+
+Garage can also be deployed on a kubernetes cluster via helm chart.
+
+## Deploying
+
+Firstly clone the repository:
+
+```bash
+git clone https://git.deuxfleurs.fr/Deuxfleurs/garage
+cd garage/scripts/helm
+```
+
+Deploy with default options:
+
+```bash
+helm install --create-namespace --namespace garage garage ./garage
+```
+
+Or deploy with custom values:
+
+```bash
+helm install --create-namespace --namespace garage garage ./garage -f values.override.yaml
+```
+
+After deploying, cluster layout must be configured manually as described in [Creating a cluster layout](@/documentation/quick-start/_index.md#creating-a-cluster-layout). Use the following command to access garage CLI:
+
+```bash
+kubectl exec --stdin --tty -n garage garage-0 -- ./garage status
+```
+
+## Overriding default values
+
+All possible configuration values can be found with:
+
+```bash
+helm show values ./garage
+```
+
+This is an example `values.overrride.yaml` for deploying in a microk8s cluster with a https s3 api ingress route:
+
+```yaml
+garage:
+ # Use only 2 replicas per object
+ replicationMode: "2"
+
+# Start 4 instances (StatefulSets) of garage
+replicaCount: 4
+
+# Override default storage class and size
+persistence:
+ meta:
+ storageClass: "openebs-hostpath"
+ size: 100Mi
+ data:
+ storageClass: "openebs-hostpath"
+ size: 1Gi
+
+ingress:
+ s3:
+ api:
+ enabled: true
+ className: "public"
+ annotations:
+ cert-manager.io/cluster-issuer: "letsencrypt-prod"
+ nginx.ingress.kubernetes.io/proxy-body-size: 500m
+ hosts:
+ - host: s3-api.my-domain.com
+ paths:
+ - path: /
+ pathType: Prefix
+ tls:
+ - secretName: garage-ingress-cert
+ hosts:
+ - s3-api.my-domain.com
+```
+
+## Removing
+
+```bash
+helm delete --namespace garage garage
+```
+
+Note that this will leave behind custom CRD `garagenodes.deuxfleurs.fr`, which must be removed manually if desired.
diff --git a/doc/book/cookbook/real-world.md b/doc/book/cookbook/real-world.md
index e101a706..4fcb5cf7 100644
--- a/doc/book/cookbook/real-world.md
+++ b/doc/book/cookbook/real-world.md
@@ -51,15 +51,15 @@ to store 2 TB of data in total.
## Get a Docker image
-Our docker image is currently named `dxflrs/amd64_garage` and is stored on the [Docker Hub](https://hub.docker.com/r/dxflrs/amd64_garage/tags?page=1&ordering=last_updated).
-We encourage you to use a fixed tag (eg. `v0.4.0`) and not the `latest` tag.
-For this example, we will use the latest published version at the time of the writing which is `v0.4.0` but it's up to you
-to check [the most recent versions on the Docker Hub](https://hub.docker.com/r/dxflrs/amd64_garage/tags?page=1&ordering=last_updated).
+Our docker image is currently named `dxflrs/garage` and is stored on the [Docker Hub](https://hub.docker.com/r/dxflrs/garage/tags?page=1&ordering=last_updated).
+We encourage you to use a fixed tag (eg. `v0.8.0`) and not the `latest` tag.
+For this example, we will use the latest published version at the time of the writing which is `v0.8.0` but it's up to you
+to check [the most recent versions on the Docker Hub](https://hub.docker.com/r/dxflrs/garage/tags?page=1&ordering=last_updated).
For example:
```
-sudo docker pull dxflrs/amd64_garage:v0.4.0
+sudo docker pull dxflrs/garage:v0.8.0
```
## Deploying and configuring Garage
@@ -125,7 +125,7 @@ docker run \
-v /etc/garage.toml:/etc/garage.toml \
-v /var/lib/garage/meta:/var/lib/garage/meta \
-v /var/lib/garage/data:/var/lib/garage/data \
- lxpz/garage_amd64:v0.4.0
+ dxflrs/garage:v0.8.0
```
It should be restarted automatically at each reboot.
diff --git a/doc/book/cookbook/reverse-proxy.md b/doc/book/cookbook/reverse-proxy.md
index 61bc7933..fb918778 100644
--- a/doc/book/cookbook/reverse-proxy.md
+++ b/doc/book/cookbook/reverse-proxy.md
@@ -100,7 +100,7 @@ server {
}
```
-## Exposing the web endpoint
+### Exposing the web endpoint
To better understand the logic involved, you can refer to the [Exposing buckets as websites](/cookbook/exposing_websites.html) section.
Otherwise, the configuration is very similar to the S3 endpoint.
@@ -140,6 +140,165 @@ server {
@TODO
-## Traefik
+## Traefik v2
+
+We will see in this part how to set up a reverse proxy with [Traefik](https://docs.traefik.io/).
+
+Here is [a basic configuration file](https://doc.traefik.io/traefik/https/acme/#configuration-examples):
+
+```toml
+[entryPoints]
+ [entryPoints.web]
+ address = ":80"
+
+ [entryPoints.websecure]
+ address = ":443"
+
+[certificatesResolvers.myresolver.acme]
+ email = "your-email@example.com"
+ storage = "acme.json"
+ [certificatesResolvers.myresolver.acme.httpChallenge]
+ # used during the challenge
+ entryPoint = "web"
+```
+
+### Add Garage service
+
+To add Garage on Traefik you should declare a new service using its IP address (or hostname) and port:
+
+```toml
+[http.services]
+ [http.services.my_garage_service.loadBalancer]
+ [[http.services.my_garage_service.loadBalancer.servers]]
+ url = "http://xxx.xxx.xxx.xxx"
+ port = 3900
+```
+
+It's possible to declare multiple Garage servers as back-ends:
+
+```toml
+[http.services]
+ [[http.services.my_garage_service.loadBalancer.servers]]
+ url = "http://xxx.xxx.xxx.xxx"
+ port = 3900
+ [[http.services.my_garage_service.loadBalancer.servers]]
+ url = "http://yyy.yyy.yyy.yyy"
+ port = 3900
+ [[http.services.my_garage_service.loadBalancer.servers]]
+ url = "http://zzz.zzz.zzz.zzz"
+ port = 3900
+```
+
+Traefik can remove unhealthy servers automatically with [a health check configuration](https://doc.traefik.io/traefik/routing/services/#health-check):
+
+```
+[http.services]
+ [http.services.my_garage_service.loadBalancer]
+ [http.services.my_garage_service.loadBalancer.healthCheck]
+ path = "/"
+ interval = "60s"
+ timeout = "5s"
+```
+
+### Adding a website
+
+To add a new website, add the following declaration to your Traefik configuration file:
+
+```toml
+[http.routers]
+ [http.routers.my_website]
+ rule = "Host(`yoururl.example.org`)"
+ service = "my_garage_service"
+ entryPoints = ["web"]
+```
+
+Enable HTTPS access to your website with the following configuration section ([documentation](https://doc.traefik.io/traefik/https/overview/)):
+
+```toml
+...
+ entryPoints = ["websecure"]
+ [http.routers.my_website.tls]
+ certResolver = "myresolver"
+...
+```
+
+### Adding gzip compression
+
+Add the following configuration section [to compress response](https://doc.traefik.io/traefik/middlewares/http/compress/) using [gzip](https://developer.mozilla.org/en-US/docs/Glossary/GZip_compression) before sending them to the client:
+
+```toml
+[http.routers]
+ [http.routers.my_website]
+ ...
+ middlewares = ["gzip_compress"]
+ ...
+[http.middlewares]
+ [http.middlewares.gzip_compress.compress]
+```
+
+### Add caching response
+
+Traefik's caching middleware is only available on [entreprise version](https://doc.traefik.io/traefik-enterprise/middlewares/http-cache/), however the freely-available [Souin plugin](https://github.com/darkweak/souin#tr%C3%A6fik-container) can also do the job. (section to be completed)
+
+### Complete example
+
+```toml
+[entryPoints]
+ [entryPoints.web]
+ address = ":80"
+
+ [entryPoints.websecure]
+ address = ":443"
+
+[certificatesResolvers.myresolver.acme]
+ email = "your-email@example.com"
+ storage = "acme.json"
+ [certificatesResolvers.myresolver.acme.httpChallenge]
+ # used during the challenge
+ entryPoint = "web"
+
+[http.routers]
+ [http.routers.my_website]
+ rule = "Host(`yoururl.example.org`)"
+ service = "my_garage_service"
+ middlewares = ["gzip_compress"]
+ entryPoints = ["websecure"]
+
+[http.services]
+ [http.services.my_garage_service.loadBalancer]
+ [http.services.my_garage_service.loadBalancer.healthCheck]
+ path = "/"
+ interval = "60s"
+ timeout = "5s"
+ [[http.services.my_garage_service.loadBalancer.servers]]
+ url = "http://xxx.xxx.xxx.xxx"
+ [[http.services.my_garage_service.loadBalancer.servers]]
+ url = "http://yyy.yyy.yyy.yyy"
+ [[http.services.my_garage_service.loadBalancer.servers]]
+ url = "http://zzz.zzz.zzz.zzz"
+
+[http.middlewares]
+ [http.middlewares.gzip_compress.compress]
+```
+
+## Caddy
+
+Your Caddy configuration can be as simple as:
+
+```caddy
+s3.garage.tld, *.s3.garage.tld {
+ reverse_proxy localhost:3900 192.168.1.2:3900 example.tld:3900
+}
+
+*.web.garage.tld {
+ reverse_proxy localhost:3902 192.168.1.2:3900 example.tld:3900
+}
+
+admin.garage.tld {
+ reverse_proxy localhost:3903
+}
+```
+
+But at the same time, the `reverse_proxy` is very flexible.
+For a production deployment, you should [read its documentation](https://caddyserver.com/docs/caddyfile/directives/reverse_proxy) as it supports features like DNS discovery of upstreams, load balancing with checks, streaming parameters, etc.
-@TODO