Write documentation on configuration file and other improvements

8 files changed, 296 insertions, 247 deletions

diff --git a/doc/book/src/getting_started/binary.md b/doc/book/src/getting_started/01_binary.md
index e48500ac..2719d959 100644
--- a/doc/book/src/getting_started/binary.md
+++ b/doc/book/src/getting_started/01_binary.md
@@ -7,14 +7,14 @@ We did not test other architecture/operating system but, as long as your archite
 ## From Docker
 
 Our docker image is currently named `lxpz/garage_amd64` and is stored on the [Docker Hub](https://hub.docker.com/r/lxpz/garage_amd64/tags?page=1&ordering=last_updated).
-We encourage you to use a fixed tag (eg. `v0.2.1`) and not the `latest` tag.
-For this example, we will use the latest published version at the time of the writing which is `v0.2.1` but it's up to you
+We encourage you to use a fixed tag (eg. `v0.3.0`) and not the `latest` tag.
+For this example, we will use the latest published version at the time of the writing which is `v0.3.0` but it's up to you
 to check [the most recent versions on the Docker Hub](https://hub.docker.com/r/lxpz/garage_amd64/tags?page=1&ordering=last_updated).
 
 For example:
 
 ```
-sudo docker pull lxpz/garage_amd64:v0.2.1
+sudo docker pull lxpz/garage_amd64:v0.3.0
 ```
 
 ## From source
diff --git a/doc/book/src/getting_started/02_test_deployment.md b/doc/book/src/getting_started/02_test_deployment.md
new file mode 100644
index 00000000..16f40dce
--- /dev/null
+++ b/doc/book/src/getting_started/02_test_deployment.md
@@ -0,0 +1,107 @@
+# Configuring a test deployment
+
+This section describes how to run a simple test Garage deployment with a single node.
+Note that this kind of deployment should not be used in production, as it provides
+no redundancy for your data!
+We will also skip intra-cluster TLS configuration, meaning that if you add nodes
+to your cluster, communication between them will not be secure.
+
+First, make sure that you have Garage installed in your command line environment.
+We will explain how to launch Garage in a Docker container, however we still
+recommend that you install the `garage` CLI on your host system in order to control
+the daemon.
+
+## Writing a first configuration file
+
+This first configuration file should allow you to get started easily with the simplest
+possible Garage deployment:
+
+```toml
+metadata_dir = "/tmp/meta"
+data_dir = "/tmp/data"
+
+replication_mode = "none"
+
+rpc_bind_addr = "[::]:3901"
+
+bootstrap_peers = []
+
+[s3_api]
+s3_region = "garage"
+api_bind_addr = "[::]:3900"
+
+[s3_web]
+bind_addr = "[::]:3902"
+root_domain = ".web.garage"
+index = "index.html"
+```
+
+Save your configuration file as `garage.toml`.
+
+As you can see in the `metadata_dir` and `data_dir` parameters, we are saving Garage's data
+in `/tmp` which gets erased when your system reboots. This means that data stored on this
+Garage server will not be persistent. Change these to locations on your HDD if you want
+your data to be persisted properly.
+
+## Launching the Garage server
+
+#### Option 1: directly (without Docker)
+
+Use the following command to launch the Garage server with our configuration file:
+
+```
+garage server -c garage.toml
+```
+
+By default, Garage displays almost no output. You can tune Garage's verbosity as follows
+(from less verbose to more verbose):
+
+```
+RUST_LOG=garage=info garage server -c garage.toml
+RUST_LOG=garage=debug garage server -c garage.toml
+RUST_LOG=garage=trace garage server -c garage.toml
+```
+
+Log level `info` is recommended for most use cases.
+Log level `debug` can help you check why your S3 API calls are not working.
+
+#### Option 2: in a Docker container
+
+Use the following command to start Garage in a docker container:
+
+```
+docker run -d \
+		-p 3901:3901 -p 3902:3902 -p 3900:3900 \
+		-v ./config.toml:/garage/config.toml \
+		lxpz/garage_amd64:v0.3.0
+```
+
+To tune Garage's verbosity level, set the `RUST_LOG` environment variable in the configuration
+at launch time. For instance:
+
+```
+docker run -d \
+		-p 3901:3901 -p 3902:3902 -p 3900:3900 \
+		-v ./config.toml:/garage/config.toml \
+		-e RUST_LOG=garage=info \
+		lxpz/garage_amd64:v0.3.0
+```
+
+## Checking that Garage runs correctly
+
+The `garage` utility is also used as a CLI tool to configure your Garage deployment.
+It tries to connect to a Garage server through the RPC protocol, by default looking
+for a Garage server at `localhost:3901`.
+
+Since our deployment already binds to port 3901, the following command should be sufficient
+to show Garage's status, provided that you installed the `garage` binary on your host system:
+
+```
+garage status
+```
+
+Move on to [controlling the Garage daemon](04_control.md) to learn more about how to
+use the Garage CLI to control your cluster.
+
+Move on to [configuring your cluster](05_cluster.md) in order to configure
+your single-node deployment for actual use!
diff --git a/doc/book/src/getting_started/03_real_world_deployment.md b/doc/book/src/getting_started/03_real_world_deployment.md
new file mode 100644
index 00000000..81b929c1
--- /dev/null
+++ b/doc/book/src/getting_started/03_real_world_deployment.md
@@ -0,0 +1,154 @@
+# Configuring a real-world Garage deployment
+
+To run Garage in cluster mode, we recommend having at least 3 nodes.
+This will allow you to setup Garage for three-way replication of your data,
+the safest and most available mode avaialble.
+
+## Generating a TLS Certificate
+
+You first need to generate TLS certificates to encrypt traffic between Garage nodes
+(reffered to as RPC traffic).
+
+To generate your TLS certificates, run on your machine:
+
+```
+wget https://git.deuxfleurs.fr/Deuxfleurs/garage/raw/branch/master/genkeys.sh
+chmod +x genkeys.sh
+./genkeys.sh
+```
+
+It will creates a folder named `pki/` containing the keys that you will used for the cluster.
+
+## Real-world deployment
+
+To run a real-world deployment, make sure you the following conditions are met:
+
+- You have at least three machines with sufficient storage space available
+
+- Each machine has a public IP address which is reachable by other machines.
+  Running behind a NAT is possible, but having several Garage nodes behind a single NAT
+  is slightly more involved as each will have to have a different RPC port number
+  (the local port number of a node must be the same as the port number exposed publicly
+  by the NAT).
+
+- Ideally, each machine should have a SSD available in addition to the HDD you are dedicating
+  to Garage. This will allow for faster access to metadata and has the potential
+  to drastically reduce Garage's response times.
+
+Before deploying garage on your infrastructure, you must inventory your machines.
+For our example, we will suppose the following infrastructure with IPv6 connectivity:
+
+| Location | Name    | IP Address | Disk Space |
+|----------|---------|------------|------------|
+| Paris    | Mercury | fc00:1::1  | 1 To       |
+| Paris    | Venus   | fc00:1::2  | 2 To       |
+| London   | Earth   | fc00:B::1  | 2 To       |
+| Brussels | Mars    | fc00:F::1  | 1.5 To     |
+
+
+On each machine, we will have a similar setup,
+especially you must consider the following folders/files:
+
+  - `/etc/garage/config.toml`: Garage daemon's configuration (see below)
+  - `/etc/garage/pki/`: Folder containing Garage certificates, must be generated on your computer and copied on the servers
+  - `/var/lib/garage/meta/`: Folder containing Garage's metadata, put this folder on a SSD if possible
+  - `/var/lib/garage/data/`: Folder containing Garage's data, this folder will grows and must be on a large storage, possibly big HDDs.
+  - `/etc/systemd/system/garage.service`: Service file to start garage at boot automatically (defined below, not required if you use docker)
+
+A valid `/etc/garage/config.toml` for our cluster would be:
+
+```toml
+metadata_dir = "/var/lib/garage/meta"
+data_dir = "/var/lib/garage/data"
+
+replication_mode = "3"
+
+rpc_bind_addr = "[::]:3901"
+
+bootstrap_peers = [
+  "[fc00:1::1]:3901",
+  "[fc00:1::2]:3901",
+  "[fc00:B::1]:3901",
+  "[fc00:F::1]:3901",
+]
+
+[rpc_tls]
+ca_cert = "/etc/garage/pki/garage-ca.crt"
+node_cert = "/etc/garage/pki/garage.crt"
+node_key = "/etc/garage/pki/garage.key"
+
+[s3_api]
+s3_region = "garage"
+api_bind_addr = "[::]:3900"
+
+[s3_web]
+bind_addr = "[::]:3902"
+root_domain = ".web.garage"
+index = "index.html"
+```
+
+Please make sure to change `bootstrap_peers` to **your** IP addresses!
+
+Check the [configuration file reference documentation](../reference_manual/configuration.md)
+to learn more about all available configuration options.
+
+### For docker users
+
+On each machine, you can run the daemon with:
+
+```bash
+docker run \
+  -d \
+  --name garaged \
+  --restart always \
+  --network host \
+  -v /etc/garage/pki:/etc/garage/pki \
+  -v /etc/garage/config.toml:/garage/config.toml \
+  -v /var/lib/garage/meta:/var/lib/garage/meta \
+  -v /var/lib/garage/data:/var/lib/garage/data \
+  lxpz/garage_amd64:v0.3.0
+```
+
+It should be restart automatically at each reboot.
+Please note that we use host networking as otherwise Docker containers
+can not communicate with IPv6.
+
+Upgrading between Garage versions should be supported transparently,
+but please check the relase notes before doing so!
+To upgrade, simply stop and remove this container and
+start again the command with a new version of garage.
+
+### For systemd/raw binary users
+
+Create a file named `/etc/systemd/system/garage.service`:
+
+```toml
+[Unit]
+Description=Garage Data Store
+After=network-online.target
+Wants=network-online.target
+
+[Service]
+Environment='RUST_LOG=garage=info' 'RUST_BACKTRACE=1'
+ExecStart=/usr/local/bin/garage server -c /etc/garage/config.toml
+
+[Install]
+WantedBy=multi-user.target
+```
+
+To start the service then automatically enable it at boot:
+
+```bash
+sudo systemctl start garage
+sudo systemctl enable garage
+```
+
+To see if the service is running and to browse its logs:
+
+```bash
+sudo systemctl status garage
+sudo journalctl -u garage
+```
+
+If you want to modify the service file, do not forget to run `systemctl daemon-reload`
+to inform `systemd` of your modifications.
diff --git a/doc/book/src/getting_started/control.md b/doc/book/src/getting_started/04_control.md
index 9a66a0dc..018d3268 100644
--- a/doc/book/src/getting_started/control.md
+++ b/doc/book/src/getting_started/04_control.md
@@ -6,8 +6,9 @@ The `garage` binary has two purposes:
 
 In this section, we will see how to use the `garage` binary as a control tool for the daemon we just started.
 You first need to get a shell having access to this binary, which depends of your configuration:
-  - with `docker-compose`, run `sudo docker-compose exec g1 bash` then `/garage/garage`
-  - with `docker`, run `sudo docker exec -ti garaged bash` then `/garage/garage`
+
+  - with `docker`, run `sudo docker exec -ti garaged bash`, you will now have a shell
+    where the Garage binary is available as `/garage/garage`
   - with `systemd`, simply run `/usr/local/bin/garage` if you followed previous instructions
 
 *You can also install the binary on your machine to remotely control the cluster.*
@@ -27,14 +28,12 @@ The 3 first ones are certificates and keys needed by TLS, the last one is simply
 Because we configure garage directly from the server, we do not need to set `--rpc-host`.
 To avoid typing the 3 first options each time we want to run a command, we will create an alias.
 
-### `docker-compose` alias
+### test deployment
+
+If you have simply deployed Garage on your local machine, without TLS, you can invoke
+`garage` directly without any of these parameters and without making a `garagectl` alias
+(replace mentions of `garagectl` in the next sections by `garage`).
 
-```bash
-alias garagectl='/garage/garage \
-  --ca-cert /pki/garage-ca.crt \
-  --client-cert /pki/garage.crt \
-  --client-key /pki/garage.key'
-```
 
 ### `docker` alias
 
@@ -45,7 +44,6 @@ alias garagectl='/garage/garage \
   --client-key /etc/garage/pki/garage.key'
 ```
 
-
 ### raw binary alias
 
 ```bash
@@ -74,4 +72,4 @@ Healthy nodes:
 8781c50c410a41b3…	758338dde686	[::ffff:172.20.0.102]:3901	UNCONFIGURED/REMOVED
 ```
 
-...which means that you are ready to configure your cluster!
+...which means that you are ready to [configure your cluster](05_cluster.md)!
diff --git a/doc/book/src/getting_started/cluster.md b/doc/book/src/getting_started/05_cluster.md
index c9c18684..83beb662 100644
--- a/doc/book/src/getting_started/cluster.md
+++ b/doc/book/src/getting_started/05_cluster.md
@@ -7,7 +7,7 @@ as well as the site (think datacenter) of each machine.
 
 ## Test cluster
 
-As this part is not relevant for a test cluster, you can use this one-liner to create a basic topology:
+As this part is not relevant for a test cluster, you can use this three-liner to create a basic topology:
 
 ```bash
 garagectl status | grep UNCONFIGURED | grep -Po '^[0-9a-f]+' | while read id; do 
@@ -19,7 +19,7 @@ done
 
 For our example, we will suppose we have the following infrastructure (Capacity, Identifier and Datacenter are specific values to garage described in the following):
 
-| Location | Name    | Disk Space | `Capacity` | `Identifier` | `Datacenter` |
+| Location | Name    | Disk Space | `Capacity` | `Identifier` | `Zone` |
 |----------|---------|------------|------------|--------------|--------------|
 | Paris    | Mercury | 1 To       | `2`        | `8781c5`     | `par1`       |
 | Paris    | Venus   | 2 To       | `4`        | `2a638e`     | `par1`       |
@@ -45,6 +45,15 @@ garagectl status
 
 It will display the IP address associated with each node; from the IP address you will be able to recognize the node.
 
+### Zones
+
+Zones are simply a user-chosen identifier that identify a group of server that are grouped together logically.
+It is up to the system administrator deploying garage to identify what does "grouped together" means.
+
+In most cases, a zone will correspond to a geographical location (i.e. a datacenter).
+Behind the scene, Garage will use zone definition to try to store the same data on different zones,
+in order to provide high availability despite failure of a zone.
+
 ### Capacity
 
 Garage reasons on an arbitrary metric about disk storage that is named the *capacity* of a node.
@@ -55,19 +64,19 @@ Additionaly, the capacity values used in Garage should be as small as possible,
 Here we chose that 1 unit of capacity = 0.5 To, so that we can express servers of size
 1 To and 2 To, as wel as the intermediate size 1.5 To.
 
-### Datacenter
-
-Datacenter are simply a user-chosen identifier that identify a group of server that are located in the same place.
-It is up to the system administrator deploying garage to identify what does "the same place" means.
-Behind the scene, garage will try to store the same data on different sites to provide high availability despite a data center failure.
+Note that the amount of data stored by Garage on each server may not be strictly proportional to
+its capacity value, as Garage will priorize having 3 copies of data in different zones,
+even if this means that capacities will not be strictly respected. For example in our above examples,
+nodes Earth and Mars will always store a copy of everything each, and the third copy will
+have 66% chance of being stored by Venus and 33% chance of being stored by Mercury.
 
 ### Inject the topology
 
 Given the information above, we will configure our cluster as follow:
 
 ```
-garagectl node configure --datacenter par1 -c 2 -t mercury 8781c5
-garagectl node configure --datacenter par1 -c 4 -t venus 2a638e
-garagectl node configure --datacenter lon1 -c 4 -t earth 68143d
-garagectl node configure --datacenter bru1 -c 3 -t mars 212f75
+garagectl node configure -z par1 -c 2 -t mercury 8781c5
+garagectl node configure -z par1 -c 4 -t venus 2a638e
+garagectl node configure -z lon1 -c 4 -t earth 68143d
+garagectl node configure -z bru1 -c 3 -t mars 212f75
 ```
diff --git a/doc/book/src/getting_started/bucket.md b/doc/book/src/getting_started/06_bucket.md
index b4a2d81d..b4a2d81d 100644
--- a/doc/book/src/getting_started/bucket.md
+++ b/doc/book/src/getting_started/06_bucket.md
diff --git a/doc/book/src/getting_started/files.md b/doc/book/src/getting_started/07_files.md
index 0e3939ce..cdd5d945 100644
--- a/doc/book/src/getting_started/files.md
+++ b/doc/book/src/getting_started/07_files.md
@@ -4,6 +4,9 @@ We recommend the use of MinIO Client to interact with Garage files (`mc`).
 Instructions to install it and use it are provided on the [MinIO website](https://docs.min.io/docs/minio-client-quickstart-guide.html).
 Before reading the following, you need a working `mc` command on your path.
 
+Note that on certain Linux distributions such as Arch Linux, the Minio client binary
+is called `mcli` instead of `mc` (to avoid name clashes with the Midnight Commander).
+
 ## Configure `mc`
 
 You need your access key and secret key created in the [previous section](bucket.md).
diff --git a/doc/book/src/getting_started/daemon.md b/doc/book/src/getting_started/daemon.md
deleted file mode 100644
index 0f45daee..00000000
--- a/doc/book/src/getting_started/daemon.md
+++ /dev/null
@@ -1,222 +0,0 @@
-# Configure the daemon
-
-Garage is a software that can be run only in a cluster and requires at least 3 instances.
-In our getting started guide, we document two deployment types:
-  - [Test deployment](#test-deployment) though `docker-compose`
-  - [Real-world deployment](#real-world-deployment) through `docker` or `systemd`
-
-In any case, you first need to generate TLS certificates, as traffic is encrypted between Garage's nodes.
-
-## Generating a TLS Certificate
-
-To generate your TLS certificates, run on your machine:
-
-```
-wget https://git.deuxfleurs.fr/Deuxfleurs/garage/raw/branch/master/genkeys.sh
-chmod +x genkeys.sh
-./genkeys.sh
-```
-
-It will creates a folder named `pki` containing the keys that you will used for the cluster.
-
-## Test deployment
-
-Single machine deployment is only described through `docker-compose`.
-
-Before starting, we recommend you create a folder for our deployment:
-
-```bash
-mkdir garage-single
-cd garage-single
-```
-
-We start by creating a file named `docker-compose.yml` describing our network and our containers:
-
-```yml
-version: '3.4'
-
-networks: { virtnet: { ipam: { config: [ subnet: 172.20.0.0/24 ]}}}
-
-services:
-  g1:
-    image: lxpz/garage_amd64:v0.1.1d
-    networks: { virtnet: { ipv4_address: 172.20.0.101 }}
-    volumes:
-      - "./pki:/pki"
-      - "./config.toml:/garage/config.toml"
-
-  g2:
-    image: lxpz/garage_amd64:v0.1.1d
-    networks: { virtnet: { ipv4_address: 172.20.0.102 }}
-    volumes:
-      - "./pki:/pki"
-      - "./config.toml:/garage/config.toml"
-
-  g3:
-    image: lxpz/garage_amd64:v0.1.1d
-    networks: { virtnet: { ipv4_address: 172.20.0.103 }}
-    volumes:
-      - "./pki:/pki"
-      - "./config.toml:/garage/config.toml"
-```
-
-*We define a static network here which is not considered as a best practise on Docker.
-The rational is that Garage only supports IP address and not domain names in its configuration, so we need to know the IP address in advance.*
-
-and then create the `config.toml` file next to it as follow:
-
-```toml
-metadata_dir = "/garage/meta"
-data_dir = "/garage/data"
-rpc_bind_addr = "[::]:3901"
-bootstrap_peers = [
-  "172.20.0.101:3901",
-  "172.20.0.102:3901",
-  "172.20.0.103:3901",
-]
-
-[rpc_tls]
-ca_cert = "/pki/garage-ca.crt"
-node_cert = "/pki/garage.crt"
-node_key = "/pki/garage.key"
-
-[s3_api]
-s3_region = "garage"
-api_bind_addr = "[::]:3900"
-
-[s3_web]
-bind_addr = "[::]:3902"
-root_domain = ".web.garage"
-index = "index.html"
-```
-
-*Please note that we have not mounted `/garage/meta` or `/garage/data` on the host: data will be lost when the container will be destroyed.*
-
-And that's all, you are ready to launch your cluster!
-
-```
-sudo docker-compose up
-```
-
-While your daemons are up, your cluster is still not configured yet.
-However, you can check that your services are still listening as expected by querying them from your host:
-
-```bash
-curl http://172.20.0.{101,102,103}:3902
-```
-
-which should give you:
-
-```
-Not found
-Not found
-Not found
-```
-
-That's all, you are ready to [configure your cluster!](./cluster.md).
-
-## Real-world deployment
-
-Before deploying garage on your infrastructure, you must inventory your machines.
-For our example, we will suppose the following infrastructure:
-
-| Location | Name    | IP Address | Disk Space |
-|----------|---------|------------|------------|
-| Paris    | Mercury | fc00:1::1  | 1 To       |
-| Paris    | Venus   | fc00:1::2  | 2 To       |
-| London   | Earth   | fc00:B::1  | 2 To       |
-| Brussels | Mars    | fc00:F::1  | 1.5 To     |
-
-On each machine, we will have a similar setup, especially you must consider the following folders/files:
-  - `/etc/garage/pki`: Garage certificates, must be generated on your computer and copied on the servers
-  - `/etc/garage/config.toml`: Garage daemon's configuration (defined below)
-  - `/etc/systemd/system/garage.service`: Service file to start garage at boot automatically (defined below, not required if you use docker)
-  - `/var/lib/garage/meta`: Contains Garage's metadata, put this folder on a SSD if possible
-  - `/var/lib/garage/data`: Contains Garage's data, this folder will grows and must be on a large storage, possibly big HDDs.
-
-A valid `/etc/garage/config.toml` for our cluster would be:
-
-```toml
-metadata_dir = "/var/lib/garage/meta"
-data_dir = "/var/lib/garage/data"
-rpc_bind_addr = "[::]:3901"
-bootstrap_peers = [
-  "[fc00:1::1]:3901",
-  "[fc00:1::2]:3901",
-  "[fc00:B::1]:3901",
-  "[fc00:F::1]:3901",
-]
-
-[rpc_tls]
-ca_cert = "/etc/garage/pki/garage-ca.crt"
-node_cert = "/etc/garage/pki/garage.crt"
-node_key = "/etc/garage/pki/garage.key"
-
-[s3_api]
-s3_region = "garage"
-api_bind_addr = "[::]:3900"
-
-[s3_web]
-bind_addr = "[::]:3902"
-root_domain = ".web.garage"
-index = "index.html"
-```
-
-Please make sure to change `bootstrap_peers` to **your** IP addresses!
-
-### For docker users
-
-On each machine, you can run the daemon with:
-
-```bash
-docker run \
-  -d \
-  --name garaged \
-  --restart always \
-  --network host \
-  -v /etc/garage/pki:/etc/garage/pki \
-  -v /etc/garage/config.toml:/garage/config.toml \
-  -v /var/lib/garage/meta:/var/lib/garage/meta \
-  -v /var/lib/garage/data:/var/lib/garage/data \
-  lxpz/garage_amd64:v0.1.1d
-```
-
-It should be restart automatically at each reboot.
-Please note that we use host networking as otherwise Docker containers can no communicate with IPv6.
-
-To upgrade, simply stop and remove this container and start again the command with a new version of garage.
-
-### For systemd/raw binary users
-
-Create a file named `/etc/systemd/system/garage.service`:
-
-```toml
-[Unit]
-Description=Garage Data Store
-After=network-online.target
-Wants=network-online.target
-
-[Service]
-Environment='RUST_LOG=garage=info' 'RUST_BACKTRACE=1'
-ExecStart=/usr/local/bin/garage server -c /etc/garage/config.toml
-
-[Install]
-WantedBy=multi-user.target
-```
-
-To start the service then automatically enable it at boot:
-
-```bash
-sudo systemctl start garage
-sudo systemctl enable garage
-```
-
-To see if the service is running and to browse its logs:
-
-```bash
-sudo systemctl status garage
-sudo journalctl -u garage
-```
-
-If you want to modify the service file, do not forget to run `systemctl daemon-reload`
-to inform `systemd` of your modifications.