diff options
Diffstat (limited to 'doc/book/src/getting_started')
| -rw-r--r-- | doc/book/src/getting_started/01_binary.md (renamed from doc/book/src/getting_started/binary.md) | 6 | ||||
| -rw-r--r-- | doc/book/src/getting_started/02_test_deployment.md | 107 | ||||
| -rw-r--r-- | doc/book/src/getting_started/03_real_world_deployment.md | 154 | ||||
| -rw-r--r-- | doc/book/src/getting_started/04_control.md (renamed from doc/book/src/getting_started/control.md) | 20 | ||||
| -rw-r--r-- | doc/book/src/getting_started/05_cluster.md (renamed from doc/book/src/getting_started/cluster.md) | 31 | ||||
| -rw-r--r-- | doc/book/src/getting_started/06_bucket.md (renamed from doc/book/src/getting_started/bucket.md) | 0 | ||||
| -rw-r--r-- | doc/book/src/getting_started/07_files.md (renamed from doc/book/src/getting_started/files.md) | 3 | ||||
| -rw-r--r-- | doc/book/src/getting_started/daemon.md | 222 |
8 files changed, 296 insertions, 247 deletions
diff --git a/doc/book/src/getting_started/binary.md b/doc/book/src/getting_started/01_binary.md index e48500ac..2719d959 100644 --- a/doc/book/src/getting_started/binary.md +++ b/doc/book/src/getting_started/01_binary.md @@ -7,14 +7,14 @@ We did not test other architecture/operating system but, as long as your archite ## From Docker Our docker image is currently named `lxpz/garage_amd64` and is stored on the [Docker Hub](https://hub.docker.com/r/lxpz/garage_amd64/tags?page=1&ordering=last_updated). -We encourage you to use a fixed tag (eg. `v0.2.1`) and not the `latest` tag. -For this example, we will use the latest published version at the time of the writing which is `v0.2.1` but it's up to you +We encourage you to use a fixed tag (eg. `v0.3.0`) and not the `latest` tag. +For this example, we will use the latest published version at the time of the writing which is `v0.3.0` but it's up to you to check [the most recent versions on the Docker Hub](https://hub.docker.com/r/lxpz/garage_amd64/tags?page=1&ordering=last_updated). For example: ``` -sudo docker pull lxpz/garage_amd64:v0.2.1 +sudo docker pull lxpz/garage_amd64:v0.3.0 ``` ## From source diff --git a/doc/book/src/getting_started/02_test_deployment.md b/doc/book/src/getting_started/02_test_deployment.md new file mode 100644 index 00000000..16f40dce --- /dev/null +++ b/doc/book/src/getting_started/02_test_deployment.md @@ -0,0 +1,107 @@ +# Configuring a test deployment + +This section describes how to run a simple test Garage deployment with a single node. +Note that this kind of deployment should not be used in production, as it provides +no redundancy for your data! +We will also skip intra-cluster TLS configuration, meaning that if you add nodes +to your cluster, communication between them will not be secure. + +First, make sure that you have Garage installed in your command line environment. +We will explain how to launch Garage in a Docker container, however we still +recommend that you install the `garage` CLI on your host system in order to control +the daemon. + +## Writing a first configuration file + +This first configuration file should allow you to get started easily with the simplest +possible Garage deployment: + +```toml +metadata_dir = "/tmp/meta" +data_dir = "/tmp/data" + +replication_mode = "none" + +rpc_bind_addr = "[::]:3901" + +bootstrap_peers = [] + +[s3_api] +s3_region = "garage" +api_bind_addr = "[::]:3900" + +[s3_web] +bind_addr = "[::]:3902" +root_domain = ".web.garage" +index = "index.html" +``` + +Save your configuration file as `garage.toml`. + +As you can see in the `metadata_dir` and `data_dir` parameters, we are saving Garage's data +in `/tmp` which gets erased when your system reboots. This means that data stored on this +Garage server will not be persistent. Change these to locations on your HDD if you want +your data to be persisted properly. + +## Launching the Garage server + +#### Option 1: directly (without Docker) + +Use the following command to launch the Garage server with our configuration file: + +``` +garage server -c garage.toml +``` + +By default, Garage displays almost no output. You can tune Garage's verbosity as follows +(from less verbose to more verbose): + +``` +RUST_LOG=garage=info garage server -c garage.toml +RUST_LOG=garage=debug garage server -c garage.toml +RUST_LOG=garage=trace garage server -c garage.toml +``` + +Log level `info` is recommended for most use cases. +Log level `debug` can help you check why your S3 API calls are not working. + +#### Option 2: in a Docker container + +Use the following command to start Garage in a docker container: + +``` +docker run -d \ + -p 3901:3901 -p 3902:3902 -p 3900:3900 \ + -v ./config.toml:/garage/config.toml \ + lxpz/garage_amd64:v0.3.0 +``` + +To tune Garage's verbosity level, set the `RUST_LOG` environment variable in the configuration +at launch time. For instance: + +``` +docker run -d \ + -p 3901:3901 -p 3902:3902 -p 3900:3900 \ + -v ./config.toml:/garage/config.toml \ + -e RUST_LOG=garage=info \ + lxpz/garage_amd64:v0.3.0 +``` + +## Checking that Garage runs correctly + +The `garage` utility is also used as a CLI tool to configure your Garage deployment. +It tries to connect to a Garage server through the RPC protocol, by default looking +for a Garage server at `localhost:3901`. + +Since our deployment already binds to port 3901, the following command should be sufficient +to show Garage's status, provided that you installed the `garage` binary on your host system: + +``` +garage status +``` + +Move on to [controlling the Garage daemon](04_control.md) to learn more about how to +use the Garage CLI to control your cluster. + +Move on to [configuring your cluster](05_cluster.md) in order to configure +your single-node deployment for actual use! diff --git a/doc/book/src/getting_started/03_real_world_deployment.md b/doc/book/src/getting_started/03_real_world_deployment.md new file mode 100644 index 00000000..81b929c1 --- /dev/null +++ b/doc/book/src/getting_started/03_real_world_deployment.md @@ -0,0 +1,154 @@ +# Configuring a real-world Garage deployment + +To run Garage in cluster mode, we recommend having at least 3 nodes. +This will allow you to setup Garage for three-way replication of your data, +the safest and most available mode avaialble. + +## Generating a TLS Certificate + +You first need to generate TLS certificates to encrypt traffic between Garage nodes +(reffered to as RPC traffic). + +To generate your TLS certificates, run on your machine: + +``` +wget https://git.deuxfleurs.fr/Deuxfleurs/garage/raw/branch/master/genkeys.sh +chmod +x genkeys.sh +./genkeys.sh +``` + +It will creates a folder named `pki/` containing the keys that you will used for the cluster. + +## Real-world deployment + +To run a real-world deployment, make sure you the following conditions are met: + +- You have at least three machines with sufficient storage space available + +- Each machine has a public IP address which is reachable by other machines. + Running behind a NAT is possible, but having several Garage nodes behind a single NAT + is slightly more involved as each will have to have a different RPC port number + (the local port number of a node must be the same as the port number exposed publicly + by the NAT). + +- Ideally, each machine should have a SSD available in addition to the HDD you are dedicating + to Garage. This will allow for faster access to metadata and has the potential + to drastically reduce Garage's response times. + +Before deploying garage on your infrastructure, you must inventory your machines. +For our example, we will suppose the following infrastructure with IPv6 connectivity: + +| Location | Name | IP Address | Disk Space | +|----------|---------|------------|------------| +| Paris | Mercury | fc00:1::1 | 1 To | +| Paris | Venus | fc00:1::2 | 2 To | +| London | Earth | fc00:B::1 | 2 To | +| Brussels | Mars | fc00:F::1 | 1.5 To | + + +On each machine, we will have a similar setup, +especially you must consider the following folders/files: + + - `/etc/garage/config.toml`: Garage daemon's configuration (see below) + - `/etc/garage/pki/`: Folder containing Garage certificates, must be generated on your computer and copied on the servers + - `/var/lib/garage/meta/`: Folder containing Garage's metadata, put this folder on a SSD if possible + - `/var/lib/garage/data/`: Folder containing Garage's data, this folder will grows and must be on a large storage, possibly big HDDs. + - `/etc/systemd/system/garage.service`: Service file to start garage at boot automatically (defined below, not required if you use docker) + +A valid `/etc/garage/config.toml` for our cluster would be: + +```toml +metadata_dir = "/var/lib/garage/meta" +data_dir = "/var/lib/garage/data" + +replication_mode = "3" + +rpc_bind_addr = "[::]:3901" + +bootstrap_peers = [ + "[fc00:1::1]:3901", + "[fc00:1::2]:3901", + "[fc00:B::1]:3901", + "[fc00:F::1]:3901", +] + +[rpc_tls] +ca_cert = "/etc/garage/pki/garage-ca.crt" +node_cert = "/etc/garage/pki/garage.crt" +node_key = "/etc/garage/pki/garage.key" + +[s3_api] +s3_region = "garage" +api_bind_addr = "[::]:3900" + +[s3_web] +bind_addr = "[::]:3902" +root_domain = ".web.garage" +index = "index.html" +``` + +Please make sure to change `bootstrap_peers` to **your** IP addresses! + +Check the [configuration file reference documentation](../reference_manual/configuration.md) +to learn more about all available configuration options. + +### For docker users + +On each machine, you can run the daemon with: + +```bash +docker run \ + -d \ + --name garaged \ + --restart always \ + --network host \ + -v /etc/garage/pki:/etc/garage/pki \ + -v /etc/garage/config.toml:/garage/config.toml \ + -v /var/lib/garage/meta:/var/lib/garage/meta \ + -v /var/lib/garage/data:/var/lib/garage/data \ + lxpz/garage_amd64:v0.3.0 +``` + +It should be restart automatically at each reboot. +Please note that we use host networking as otherwise Docker containers +can not communicate with IPv6. + +Upgrading between Garage versions should be supported transparently, +but please check the relase notes before doing so! +To upgrade, simply stop and remove this container and +start again the command with a new version of garage. + +### For systemd/raw binary users + +Create a file named `/etc/systemd/system/garage.service`: + +```toml +[Unit] +Description=Garage Data Store +After=network-online.target +Wants=network-online.target + +[Service] +Environment='RUST_LOG=garage=info' 'RUST_BACKTRACE=1' +ExecStart=/usr/local/bin/garage server -c /etc/garage/config.toml + +[Install] +WantedBy=multi-user.target +``` + +To start the service then automatically enable it at boot: + +```bash +sudo systemctl start garage +sudo systemctl enable garage +``` + +To see if the service is running and to browse its logs: + +```bash +sudo systemctl status garage +sudo journalctl -u garage +``` + +If you want to modify the service file, do not forget to run `systemctl daemon-reload` +to inform `systemd` of your modifications. diff --git a/doc/book/src/getting_started/control.md b/doc/book/src/getting_started/04_control.md index 9a66a0dc..018d3268 100644 --- a/doc/book/src/getting_started/control.md +++ b/doc/book/src/getting_started/04_control.md @@ -6,8 +6,9 @@ The `garage` binary has two purposes: In this section, we will see how to use the `garage` binary as a control tool for the daemon we just started. You first need to get a shell having access to this binary, which depends of your configuration: - - with `docker-compose`, run `sudo docker-compose exec g1 bash` then `/garage/garage` - - with `docker`, run `sudo docker exec -ti garaged bash` then `/garage/garage` + + - with `docker`, run `sudo docker exec -ti garaged bash`, you will now have a shell + where the Garage binary is available as `/garage/garage` - with `systemd`, simply run `/usr/local/bin/garage` if you followed previous instructions *You can also install the binary on your machine to remotely control the cluster.* @@ -27,14 +28,12 @@ The 3 first ones are certificates and keys needed by TLS, the last one is simply Because we configure garage directly from the server, we do not need to set `--rpc-host`. To avoid typing the 3 first options each time we want to run a command, we will create an alias. -### `docker-compose` alias +### test deployment + +If you have simply deployed Garage on your local machine, without TLS, you can invoke +`garage` directly without any of these parameters and without making a `garagectl` alias +(replace mentions of `garagectl` in the next sections by `garage`). -```bash -alias garagectl='/garage/garage \ - --ca-cert /pki/garage-ca.crt \ - --client-cert /pki/garage.crt \ - --client-key /pki/garage.key' -``` ### `docker` alias @@ -45,7 +44,6 @@ alias garagectl='/garage/garage \ --client-key /etc/garage/pki/garage.key' ``` - ### raw binary alias ```bash @@ -74,4 +72,4 @@ Healthy nodes: 8781c50c410a41b3… 758338dde686 [::ffff:172.20.0.102]:3901 UNCONFIGURED/REMOVED ``` -...which means that you are ready to configure your cluster! +...which means that you are ready to [configure your cluster](05_cluster.md)! diff --git a/doc/book/src/getting_started/cluster.md b/doc/book/src/getting_started/05_cluster.md index c9c18684..83beb662 100644 --- a/doc/book/src/getting_started/cluster.md +++ b/doc/book/src/getting_started/05_cluster.md @@ -7,7 +7,7 @@ as well as the site (think datacenter) of each machine. ## Test cluster -As this part is not relevant for a test cluster, you can use this one-liner to create a basic topology: +As this part is not relevant for a test cluster, you can use this three-liner to create a basic topology: ```bash garagectl status | grep UNCONFIGURED | grep -Po '^[0-9a-f]+' | while read id; do @@ -19,7 +19,7 @@ done For our example, we will suppose we have the following infrastructure (Capacity, Identifier and Datacenter are specific values to garage described in the following): -| Location | Name | Disk Space | `Capacity` | `Identifier` | `Datacenter` | +| Location | Name | Disk Space | `Capacity` | `Identifier` | `Zone` | |----------|---------|------------|------------|--------------|--------------| | Paris | Mercury | 1 To | `2` | `8781c5` | `par1` | | Paris | Venus | 2 To | `4` | `2a638e` | `par1` | @@ -45,6 +45,15 @@ garagectl status It will display the IP address associated with each node; from the IP address you will be able to recognize the node. +### Zones + +Zones are simply a user-chosen identifier that identify a group of server that are grouped together logically. +It is up to the system administrator deploying garage to identify what does "grouped together" means. + +In most cases, a zone will correspond to a geographical location (i.e. a datacenter). +Behind the scene, Garage will use zone definition to try to store the same data on different zones, +in order to provide high availability despite failure of a zone. + ### Capacity Garage reasons on an arbitrary metric about disk storage that is named the *capacity* of a node. @@ -55,19 +64,19 @@ Additionaly, the capacity values used in Garage should be as small as possible, Here we chose that 1 unit of capacity = 0.5 To, so that we can express servers of size 1 To and 2 To, as wel as the intermediate size 1.5 To. -### Datacenter - -Datacenter are simply a user-chosen identifier that identify a group of server that are located in the same place. -It is up to the system administrator deploying garage to identify what does "the same place" means. -Behind the scene, garage will try to store the same data on different sites to provide high availability despite a data center failure. +Note that the amount of data stored by Garage on each server may not be strictly proportional to +its capacity value, as Garage will priorize having 3 copies of data in different zones, +even if this means that capacities will not be strictly respected. For example in our above examples, +nodes Earth and Mars will always store a copy of everything each, and the third copy will +have 66% chance of being stored by Venus and 33% chance of being stored by Mercury. ### Inject the topology Given the information above, we will configure our cluster as follow: ``` -garagectl node configure --datacenter par1 -c 2 -t mercury 8781c5 -garagectl node configure --datacenter par1 -c 4 -t venus 2a638e -garagectl node configure --datacenter lon1 -c 4 -t earth 68143d -garagectl node configure --datacenter bru1 -c 3 -t mars 212f75 +garagectl node configure -z par1 -c 2 -t mercury 8781c5 +garagectl node configure -z par1 -c 4 -t venus 2a638e +garagectl node configure -z lon1 -c 4 -t earth 68143d +garagectl node configure -z bru1 -c 3 -t mars 212f75 ``` diff --git a/doc/book/src/getting_started/bucket.md b/doc/book/src/getting_started/06_bucket.md index b4a2d81d..b4a2d81d 100644 --- a/doc/book/src/getting_started/bucket.md +++ b/doc/book/src/getting_started/06_bucket.md diff --git a/doc/book/src/getting_started/files.md b/doc/book/src/getting_started/07_files.md index 0e3939ce..cdd5d945 100644 --- a/doc/book/src/getting_started/files.md +++ b/doc/book/src/getting_started/07_files.md @@ -4,6 +4,9 @@ We recommend the use of MinIO Client to interact with Garage files (`mc`). Instructions to install it and use it are provided on the [MinIO website](https://docs.min.io/docs/minio-client-quickstart-guide.html). Before reading the following, you need a working `mc` command on your path. +Note that on certain Linux distributions such as Arch Linux, the Minio client binary +is called `mcli` instead of `mc` (to avoid name clashes with the Midnight Commander). + ## Configure `mc` You need your access key and secret key created in the [previous section](bucket.md). diff --git a/doc/book/src/getting_started/daemon.md b/doc/book/src/getting_started/daemon.md deleted file mode 100644 index 0f45daee..00000000 --- a/doc/book/src/getting_started/daemon.md +++ /dev/null @@ -1,222 +0,0 @@ -# Configure the daemon - -Garage is a software that can be run only in a cluster and requires at least 3 instances. -In our getting started guide, we document two deployment types: - - [Test deployment](#test-deployment) though `docker-compose` - - [Real-world deployment](#real-world-deployment) through `docker` or `systemd` - -In any case, you first need to generate TLS certificates, as traffic is encrypted between Garage's nodes. - -## Generating a TLS Certificate - -To generate your TLS certificates, run on your machine: - -``` -wget https://git.deuxfleurs.fr/Deuxfleurs/garage/raw/branch/master/genkeys.sh -chmod +x genkeys.sh -./genkeys.sh -``` - -It will creates a folder named `pki` containing the keys that you will used for the cluster. - -## Test deployment - -Single machine deployment is only described through `docker-compose`. - -Before starting, we recommend you create a folder for our deployment: - -```bash -mkdir garage-single -cd garage-single -``` - -We start by creating a file named `docker-compose.yml` describing our network and our containers: - -```yml -version: '3.4' - -networks: { virtnet: { ipam: { config: [ subnet: 172.20.0.0/24 ]}}} - -services: - g1: - image: lxpz/garage_amd64:v0.1.1d - networks: { virtnet: { ipv4_address: 172.20.0.101 }} - volumes: - - "./pki:/pki" - - "./config.toml:/garage/config.toml" - - g2: - image: lxpz/garage_amd64:v0.1.1d - networks: { virtnet: { ipv4_address: 172.20.0.102 }} - volumes: - - "./pki:/pki" - - "./config.toml:/garage/config.toml" - - g3: - image: lxpz/garage_amd64:v0.1.1d - networks: { virtnet: { ipv4_address: 172.20.0.103 }} - volumes: - - "./pki:/pki" - - "./config.toml:/garage/config.toml" -``` - -*We define a static network here which is not considered as a best practise on Docker. -The rational is that Garage only supports IP address and not domain names in its configuration, so we need to know the IP address in advance.* - -and then create the `config.toml` file next to it as follow: - -```toml -metadata_dir = "/garage/meta" -data_dir = "/garage/data" -rpc_bind_addr = "[::]:3901" -bootstrap_peers = [ - "172.20.0.101:3901", - "172.20.0.102:3901", - "172.20.0.103:3901", -] - -[rpc_tls] -ca_cert = "/pki/garage-ca.crt" -node_cert = "/pki/garage.crt" -node_key = "/pki/garage.key" - -[s3_api] -s3_region = "garage" -api_bind_addr = "[::]:3900" - -[s3_web] -bind_addr = "[::]:3902" -root_domain = ".web.garage" -index = "index.html" -``` - -*Please note that we have not mounted `/garage/meta` or `/garage/data` on the host: data will be lost when the container will be destroyed.* - -And that's all, you are ready to launch your cluster! - -``` -sudo docker-compose up -``` - -While your daemons are up, your cluster is still not configured yet. -However, you can check that your services are still listening as expected by querying them from your host: - -```bash -curl http://172.20.0.{101,102,103}:3902 -``` - -which should give you: - -``` -Not found -Not found -Not found -``` - -That's all, you are ready to [configure your cluster!](./cluster.md). - -## Real-world deployment - -Before deploying garage on your infrastructure, you must inventory your machines. -For our example, we will suppose the following infrastructure: - -| Location | Name | IP Address | Disk Space | -|----------|---------|------------|------------| -| Paris | Mercury | fc00:1::1 | 1 To | -| Paris | Venus | fc00:1::2 | 2 To | -| London | Earth | fc00:B::1 | 2 To | -| Brussels | Mars | fc00:F::1 | 1.5 To | - -On each machine, we will have a similar setup, especially you must consider the following folders/files: - - `/etc/garage/pki`: Garage certificates, must be generated on your computer and copied on the servers - - `/etc/garage/config.toml`: Garage daemon's configuration (defined below) - - `/etc/systemd/system/garage.service`: Service file to start garage at boot automatically (defined below, not required if you use docker) - - `/var/lib/garage/meta`: Contains Garage's metadata, put this folder on a SSD if possible - - `/var/lib/garage/data`: Contains Garage's data, this folder will grows and must be on a large storage, possibly big HDDs. - -A valid `/etc/garage/config.toml` for our cluster would be: - -```toml -metadata_dir = "/var/lib/garage/meta" -data_dir = "/var/lib/garage/data" -rpc_bind_addr = "[::]:3901" -bootstrap_peers = [ - "[fc00:1::1]:3901", - "[fc00:1::2]:3901", - "[fc00:B::1]:3901", - "[fc00:F::1]:3901", -] - -[rpc_tls] -ca_cert = "/etc/garage/pki/garage-ca.crt" -node_cert = "/etc/garage/pki/garage.crt" -node_key = "/etc/garage/pki/garage.key" - -[s3_api] -s3_region = "garage" -api_bind_addr = "[::]:3900" - -[s3_web] -bind_addr = "[::]:3902" -root_domain = ".web.garage" -index = "index.html" -``` - -Please make sure to change `bootstrap_peers` to **your** IP addresses! - -### For docker users - -On each machine, you can run the daemon with: - -```bash -docker run \ - -d \ - --name garaged \ - --restart always \ - --network host \ - -v /etc/garage/pki:/etc/garage/pki \ - -v /etc/garage/config.toml:/garage/config.toml \ - -v /var/lib/garage/meta:/var/lib/garage/meta \ - -v /var/lib/garage/data:/var/lib/garage/data \ - lxpz/garage_amd64:v0.1.1d -``` - -It should be restart automatically at each reboot. -Please note that we use host networking as otherwise Docker containers can no communicate with IPv6. - -To upgrade, simply stop and remove this container and start again the command with a new version of garage. - -### For systemd/raw binary users - -Create a file named `/etc/systemd/system/garage.service`: - -```toml -[Unit] -Description=Garage Data Store -After=network-online.target -Wants=network-online.target - -[Service] -Environment='RUST_LOG=garage=info' 'RUST_BACKTRACE=1' -ExecStart=/usr/local/bin/garage server -c /etc/garage/config.toml - -[Install] -WantedBy=multi-user.target -``` - -To start the service then automatically enable it at boot: - -```bash -sudo systemctl start garage -sudo systemctl enable garage -``` - -To see if the service is running and to browse its logs: - -```bash -sudo systemctl status garage -sudo journalctl -u garage -``` - -If you want to modify the service file, do not forget to run `systemctl daemon-reload` -to inform `systemd` of your modifications. |