diff options
Diffstat (limited to 'doc/book/cookbook')
-rw-r--r-- | doc/book/cookbook/exposing-websites.md | 4 | ||||
-rw-r--r-- | doc/book/cookbook/from-source.md | 74 | ||||
-rw-r--r-- | doc/book/cookbook/kubernetes.md | 87 | ||||
-rw-r--r-- | doc/book/cookbook/real-world.md | 12 | ||||
-rw-r--r-- | doc/book/cookbook/reverse-proxy.md | 165 |
5 files changed, 313 insertions, 29 deletions
diff --git a/doc/book/cookbook/exposing-websites.md b/doc/book/cookbook/exposing-websites.md index be462dc9..5f6a5a28 100644 --- a/doc/book/cookbook/exposing-websites.md +++ b/doc/book/cookbook/exposing-websites.md @@ -5,12 +5,14 @@ weight = 25 ## Configuring a bucket for website access -There are two methods to expose buckets as website: +There are three methods to expose buckets as website: 1. using the PutBucketWebsite S3 API call, which is allowed for access keys that have the owner permission bit set 2. from the Garage CLI, by an adminstrator of the cluster +3. using the Garage administration API + The `PutBucketWebsite` API endpoint [is documented](https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutBucketWebsite.html) in the official AWS docs. This endpoint can also be called [using `aws s3api`](https://docs.aws.amazon.com/cli/latest/reference/s3api/put-bucket-website.html) on the command line. The website configuration supported by Garage is only a subset of the possibilities on Amazon S3: redirections are not supported, only the index document and error document can be specified. diff --git a/doc/book/cookbook/from-source.md b/doc/book/cookbook/from-source.md index 5973d411..bacf93ab 100644 --- a/doc/book/cookbook/from-source.md +++ b/doc/book/cookbook/from-source.md @@ -20,40 +20,76 @@ sudo apt-get update sudo apt-get install build-essential ``` -## Using source from `crates.io` +## Building from source from the Gitea repository -Garage's source code is published on `crates.io`, Rust's official package repository. -This means you can simply ask `cargo` to download and build this source code for you: +The primary location for Garage's source code is the +[Gitea repository](https://git.deuxfleurs.fr/Deuxfleurs/garage), +which contains all of the released versions as well as the code +for the developpement of the next version. + +Clone the repository and enter it as follows: ```bash -cargo install garage +git clone https://git.deuxfleurs.fr/Deuxfleurs/garage.git +cd garage ``` -That's all, `garage` should be in `$HOME/.cargo/bin`. - -You can add this folder to your `$PATH` or copy the binary somewhere else on your system. -For instance: +If you wish to build a specific version of Garage, check out the corresponding tag. For instance: ```bash -sudo cp $HOME/.cargo/bin/garage /usr/local/bin/garage +git tag # List available tags +git checkout v0.8.0 # Change v0.8.0 with the version you wish to build ``` +Otherwise you will be building a developpement build from the `main` branch +that includes all of the changes to be released in the next version. +Be careful that such a build might be unstable or contain bugs, +and could be incompatible with nodes that run stable versions of Garage. -## Using source from the Gitea repository +Finally, build Garage with the following command: -The primary location for Garage's source code is the -[Gitea repository](https://git.deuxfleurs.fr/Deuxfleurs/garage). +```bash +cargo build --release +``` -Clone the repository and build Garage with the following commands: +The binary built this way can now be found in `target/release/garage`. +You may simply copy this binary to somewhere in your `$PATH` in order to +have the `garage` command available in your shell, for instance: ```bash -git clone https://git.deuxfleurs.fr/Deuxfleurs/garage.git -cd garage -cargo build +sudo cp target/release/garage /usr/local/bin/garage ``` -Be careful, as this will make a debug build of Garage, which will be extremely slow! -To make a release build, invoke `cargo build --release` (this takes much longer). +If you are planning to develop Garage, +you might be interested in producing debug builds, which compile faster but run slower: +this can be done by removing the `--release` flag, and the resulting build can then +be found in `target/debug/garage`. + +## List of available Cargo feature flags -The binaries built this way are found in `target/{debug,release}/garage`. +Garage supports a number of compilation options in the form of Cargo feature flags, +which can be used to provide builds adapted to your system and your use case. +To produce a build with a given set of features, invoke the `cargo build` command +as follows: + +```bash +# This will build the default feature set plus feature1, feature2 and feature3 +cargo build --release --features feature1,feature2,feature3 +# This will build ONLY feature1, feature2 and feature3 +cargo build --release --no-default-features \ + --features feature1,feature2,feature3 +``` +The following feature flags are available in v0.8.0: + +| Feature flag | Enabled | Description | +| ------------ | ------- | ----------- | +| `bundled-libs` | *by default* | Use bundled version of sqlite3, zstd, lmdb and libsodium | +| `system-libs` | optional | Use system version of sqlite3, zstd, lmdb and libsodium<br>if available (exclusive with `bundled-libs`, build using<br>`cargo build --no-default-features --features system-libs`) | +| `k2v` | optional | Enable the experimental K2V API (if used, all nodes on your<br>Garage cluster must have it enabled as well) | +| `kubernetes-discovery` | optional | Enable automatic registration and discovery<br>of cluster nodes through the Kubernetes API | +| `metrics` | *by default* | Enable collection of metrics in Prometheus format on the admin API | +| `telemetry-otlp` | optional | Enable collection of execution traces using OpenTelemetry | +| `sled` | *by default* | Enable using Sled to store Garage's metadata | +| `lmdb` | optional | Enable using LMDB to store Garage's metadata | +| `sqlite` | optional | Enable using Sqlite3 to store Garage's metadata | diff --git a/doc/book/cookbook/kubernetes.md b/doc/book/cookbook/kubernetes.md new file mode 100644 index 00000000..9eafe3e1 --- /dev/null +++ b/doc/book/cookbook/kubernetes.md @@ -0,0 +1,87 @@ ++++ +title = "Deploying on Kubernetes" +weight = 32 ++++ + +Garage can also be deployed on a kubernetes cluster via helm chart. + +## Deploying + +Firstly clone the repository: + +```bash +git clone https://git.deuxfleurs.fr/Deuxfleurs/garage +cd garage/scripts/helm +``` + +Deploy with default options: + +```bash +helm install --create-namespace --namespace garage garage ./garage +``` + +Or deploy with custom values: + +```bash +helm install --create-namespace --namespace garage garage ./garage -f values.override.yaml +``` + +After deploying, cluster layout must be configured manually as described in [Creating a cluster layout](@/documentation/quick-start/_index.md#creating-a-cluster-layout). Use the following command to access garage CLI: + +```bash +kubectl exec --stdin --tty -n garage garage-0 -- ./garage status +``` + +## Overriding default values + +All possible configuration values can be found with: + +```bash +helm show values ./garage +``` + +This is an example `values.overrride.yaml` for deploying in a microk8s cluster with a https s3 api ingress route: + +```yaml +garage: + # Use only 2 replicas per object + replicationMode: "2" + +# Start 4 instances (StatefulSets) of garage +replicaCount: 4 + +# Override default storage class and size +persistence: + meta: + storageClass: "openebs-hostpath" + size: 100Mi + data: + storageClass: "openebs-hostpath" + size: 1Gi + +ingress: + s3: + api: + enabled: true + className: "public" + annotations: + cert-manager.io/cluster-issuer: "letsencrypt-prod" + nginx.ingress.kubernetes.io/proxy-body-size: 500m + hosts: + - host: s3-api.my-domain.com + paths: + - path: / + pathType: Prefix + tls: + - secretName: garage-ingress-cert + hosts: + - s3-api.my-domain.com +``` + +## Removing + +```bash +helm delete --namespace garage garage +``` + +Note that this will leave behind custom CRD `garagenodes.deuxfleurs.fr`, which must be removed manually if desired. diff --git a/doc/book/cookbook/real-world.md b/doc/book/cookbook/real-world.md index e101a706..4fcb5cf7 100644 --- a/doc/book/cookbook/real-world.md +++ b/doc/book/cookbook/real-world.md @@ -51,15 +51,15 @@ to store 2 TB of data in total. ## Get a Docker image -Our docker image is currently named `dxflrs/amd64_garage` and is stored on the [Docker Hub](https://hub.docker.com/r/dxflrs/amd64_garage/tags?page=1&ordering=last_updated). -We encourage you to use a fixed tag (eg. `v0.4.0`) and not the `latest` tag. -For this example, we will use the latest published version at the time of the writing which is `v0.4.0` but it's up to you -to check [the most recent versions on the Docker Hub](https://hub.docker.com/r/dxflrs/amd64_garage/tags?page=1&ordering=last_updated). +Our docker image is currently named `dxflrs/garage` and is stored on the [Docker Hub](https://hub.docker.com/r/dxflrs/garage/tags?page=1&ordering=last_updated). +We encourage you to use a fixed tag (eg. `v0.8.0`) and not the `latest` tag. +For this example, we will use the latest published version at the time of the writing which is `v0.8.0` but it's up to you +to check [the most recent versions on the Docker Hub](https://hub.docker.com/r/dxflrs/garage/tags?page=1&ordering=last_updated). For example: ``` -sudo docker pull dxflrs/amd64_garage:v0.4.0 +sudo docker pull dxflrs/garage:v0.8.0 ``` ## Deploying and configuring Garage @@ -125,7 +125,7 @@ docker run \ -v /etc/garage.toml:/etc/garage.toml \ -v /var/lib/garage/meta:/var/lib/garage/meta \ -v /var/lib/garage/data:/var/lib/garage/data \ - lxpz/garage_amd64:v0.4.0 + dxflrs/garage:v0.8.0 ``` It should be restarted automatically at each reboot. diff --git a/doc/book/cookbook/reverse-proxy.md b/doc/book/cookbook/reverse-proxy.md index 61bc7933..fb918778 100644 --- a/doc/book/cookbook/reverse-proxy.md +++ b/doc/book/cookbook/reverse-proxy.md @@ -100,7 +100,7 @@ server { } ``` -## Exposing the web endpoint +### Exposing the web endpoint To better understand the logic involved, you can refer to the [Exposing buckets as websites](/cookbook/exposing_websites.html) section. Otherwise, the configuration is very similar to the S3 endpoint. @@ -140,6 +140,165 @@ server { @TODO -## Traefik +## Traefik v2 + +We will see in this part how to set up a reverse proxy with [Traefik](https://docs.traefik.io/). + +Here is [a basic configuration file](https://doc.traefik.io/traefik/https/acme/#configuration-examples): + +```toml +[entryPoints] + [entryPoints.web] + address = ":80" + + [entryPoints.websecure] + address = ":443" + +[certificatesResolvers.myresolver.acme] + email = "your-email@example.com" + storage = "acme.json" + [certificatesResolvers.myresolver.acme.httpChallenge] + # used during the challenge + entryPoint = "web" +``` + +### Add Garage service + +To add Garage on Traefik you should declare a new service using its IP address (or hostname) and port: + +```toml +[http.services] + [http.services.my_garage_service.loadBalancer] + [[http.services.my_garage_service.loadBalancer.servers]] + url = "http://xxx.xxx.xxx.xxx" + port = 3900 +``` + +It's possible to declare multiple Garage servers as back-ends: + +```toml +[http.services] + [[http.services.my_garage_service.loadBalancer.servers]] + url = "http://xxx.xxx.xxx.xxx" + port = 3900 + [[http.services.my_garage_service.loadBalancer.servers]] + url = "http://yyy.yyy.yyy.yyy" + port = 3900 + [[http.services.my_garage_service.loadBalancer.servers]] + url = "http://zzz.zzz.zzz.zzz" + port = 3900 +``` + +Traefik can remove unhealthy servers automatically with [a health check configuration](https://doc.traefik.io/traefik/routing/services/#health-check): + +``` +[http.services] + [http.services.my_garage_service.loadBalancer] + [http.services.my_garage_service.loadBalancer.healthCheck] + path = "/" + interval = "60s" + timeout = "5s" +``` + +### Adding a website + +To add a new website, add the following declaration to your Traefik configuration file: + +```toml +[http.routers] + [http.routers.my_website] + rule = "Host(`yoururl.example.org`)" + service = "my_garage_service" + entryPoints = ["web"] +``` + +Enable HTTPS access to your website with the following configuration section ([documentation](https://doc.traefik.io/traefik/https/overview/)): + +```toml +... + entryPoints = ["websecure"] + [http.routers.my_website.tls] + certResolver = "myresolver" +... +``` + +### Adding gzip compression + +Add the following configuration section [to compress response](https://doc.traefik.io/traefik/middlewares/http/compress/) using [gzip](https://developer.mozilla.org/en-US/docs/Glossary/GZip_compression) before sending them to the client: + +```toml +[http.routers] + [http.routers.my_website] + ... + middlewares = ["gzip_compress"] + ... +[http.middlewares] + [http.middlewares.gzip_compress.compress] +``` + +### Add caching response + +Traefik's caching middleware is only available on [entreprise version](https://doc.traefik.io/traefik-enterprise/middlewares/http-cache/), however the freely-available [Souin plugin](https://github.com/darkweak/souin#tr%C3%A6fik-container) can also do the job. (section to be completed) + +### Complete example + +```toml +[entryPoints] + [entryPoints.web] + address = ":80" + + [entryPoints.websecure] + address = ":443" + +[certificatesResolvers.myresolver.acme] + email = "your-email@example.com" + storage = "acme.json" + [certificatesResolvers.myresolver.acme.httpChallenge] + # used during the challenge + entryPoint = "web" + +[http.routers] + [http.routers.my_website] + rule = "Host(`yoururl.example.org`)" + service = "my_garage_service" + middlewares = ["gzip_compress"] + entryPoints = ["websecure"] + +[http.services] + [http.services.my_garage_service.loadBalancer] + [http.services.my_garage_service.loadBalancer.healthCheck] + path = "/" + interval = "60s" + timeout = "5s" + [[http.services.my_garage_service.loadBalancer.servers]] + url = "http://xxx.xxx.xxx.xxx" + [[http.services.my_garage_service.loadBalancer.servers]] + url = "http://yyy.yyy.yyy.yyy" + [[http.services.my_garage_service.loadBalancer.servers]] + url = "http://zzz.zzz.zzz.zzz" + +[http.middlewares] + [http.middlewares.gzip_compress.compress] +``` + +## Caddy + +Your Caddy configuration can be as simple as: + +```caddy +s3.garage.tld, *.s3.garage.tld { + reverse_proxy localhost:3900 192.168.1.2:3900 example.tld:3900 +} + +*.web.garage.tld { + reverse_proxy localhost:3902 192.168.1.2:3900 example.tld:3900 +} + +admin.garage.tld { + reverse_proxy localhost:3903 +} +``` + +But at the same time, the `reverse_proxy` is very flexible. +For a production deployment, you should [read its documentation](https://caddyserver.com/docs/caddyfile/directives/reverse_proxy) as it supports features like DNS discovery of upstreams, load balancing with checks, streaming parameters, etc. -@TODO |