Merge pull request 'Add a `mdbook` documentation to present garage and help user on-boarding' (#45) from feature/mdbook into master

Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/45

7 files changed, 540 insertions, 0 deletions

diff --git a/doc/book/src/getting_started/binary.md b/doc/book/src/getting_started/binary.md
new file mode 100644
index 00000000..9a18babc
--- /dev/null
+++ b/doc/book/src/getting_started/binary.md
@@ -0,0 +1,44 @@
+# Get a binary
+
+Currently, only two installations procedures are supported for Garage: from Docker (x86\_64 for Linux) and from source.
+In the future, we plan to add a third one, by publishing a compiled binary (x86\_64 for Linux).
+We did not test other architecture/operating system but, as long as your architecture/operating system is supported by Rust, you should be able to run Garage (feel free to report your tests!).
+
+## From Docker
+
+Our docker image is currently named `lxpz/garage_amd64` and is stored on the [Docker Hub](https://hub.docker.com/r/lxpz/garage_amd64/tags?page=1&ordering=last_updated).
+We encourage you to use a fixed tag (eg. `v0.1.1d`) and not the `latest` tag.
+For this example, we will use the latest published version at the time of the writing which is `v0.1.1d` but it's up to you
+to check [the most recent versions on the Docker Hub](https://hub.docker.com/r/lxpz/garage_amd64/tags?page=1&ordering=last_updated).
+
+For example:
+
+```
+sudo docker pull lxpz/garage_amd64:v0.1.1d
+```
+
+## From source
+
+Garage is a standard Rust project.
+First, you need `rust` and `cargo`.
+On Debian:
+
+```bash
+sudo apt-get update
+sudo apt-get install -y rustc cargo
+```
+
+Then, you can ask cargo to install the binary for you:
+
+```bash
+cargo install garage
+```
+
+That's all, `garage` should be in `$HOME/.cargo/bin`.
+You can add this folder to your `$PATH` or copy the binary somewhere else on your system.
+For the following, we will assume you copied it in `/usr/local/bin/garage`:
+
+```bash
+sudo cp $HOME/.cargo/bin/garage /usr/local/bin/garage
+```
+
diff --git a/doc/book/src/getting_started/bucket.md b/doc/book/src/getting_started/bucket.md
new file mode 100644
index 00000000..b22ce788
--- /dev/null
+++ b/doc/book/src/getting_started/bucket.md
@@ -0,0 +1,78 @@
+# Create buckets and keys
+
+*We use a command named `garagectl` which is in fact an alias you must define as explained in the [Control the daemon](./daemon.md) section.*
+
+In this section, we will suppose that we want to create a bucket named `nextcloud-bucket`
+that will be accessed through a key named `nextcloud-app-key`.
+
+Don't forget that `help` command and `--help` subcommands can help you anywhere, the CLI tool is self-documented! Two examples:
+
+```
+garagectl help
+garagectl bucket allow --help
+```
+
+## Create a bucket
+
+Fine, now let's create a bucket (we imagine that you want to deploy nextcloud):
+
+```
+garagectl bucket create nextcloud-bucket
+```
+
+Check that everything went well:
+
+```
+garagectl bucket list
+garagectl bucket info nextcloud-bucket
+```
+
+## Create an API key
+
+Now we will generate an API key to access this bucket.
+Note that API keys are independent of buckets: one key can access multiple buckets, multiple keys can access one bucket.
+
+Now, let's start by creating a key only for our PHP application:
+
+```
+garagectl key new --name nextcloud-app-key
+```
+
+You will have the following output (this one is fake, `key_id` and `secret_key` were generated with the openssl CLI tool):
+
+```javascript
+Key { 
+  key_id: "GK3515373e4c851ebaad366558", 
+  secret_key: "7d37d093435a41f2aab8f13c19ba067d9776c90215f56614adad6ece597dbb34", 
+  name: "nextcloud-app-key", 
+  name_timestamp: 1603280506694, 
+  deleted: false, 
+  authorized_buckets: [] 
+}
+```
+
+Check that everything works as intended (be careful, info works only with your key identifier and not with its friendly name!):
+
+```
+garagectl key list
+garagectl key info GK3515373e4c851ebaad366558
+```
+
+## Allow a key to access a bucket
+
+Now that we have a bucket and a key, we need to give permissions to the key on the bucket!
+
+```
+garagectl bucket allow \
+  --read \
+  --write 
+  nextcloud-bucket \
+  --key GK3515373e4c851ebaad366558
+```
+
+You can check at any times allowed keys on your bucket with:
+
+```
+garagectl bucket info nextcloud-bucket
+```
+
diff --git a/doc/book/src/getting_started/cluster.md b/doc/book/src/getting_started/cluster.md
new file mode 100644
index 00000000..af6e8f10
--- /dev/null
+++ b/doc/book/src/getting_started/cluster.md
@@ -0,0 +1,72 @@
+# Configure a cluster
+
+*We use a command named `garagectl` which is in fact an alias you must define as explained in the [Control the daemon](./daemon.md) section.*
+
+In this section, we will inform garage of the disk space available on each node of the cluster
+as well as the site (think datacenter) of each machine.
+
+## Test cluster
+
+As this part is not relevant for a test cluster, you can use this one-liner to create a basic topology:
+
+```bash
+garagectl status | grep UNCONFIGURED | grep -Po '^[0-9a-f]+' | while read id; do 
+    garagectl node configure -d dc1 -n 10 $id
+done
+```
+
+## Real-world cluster
+
+For our example, we will suppose we have the following infrastructure (Tokens, Identifier and Datacenter are specific values to garage described in the following):
+
+| Location | Name    | Disk Space | `Tokens` | `Identifier` | `Datacenter` |
+|----------|---------|------------|----------|--------------|--------------|
+| Paris    | Mercury | 1 To       | `100`    | `8781c5`     | `par1`       |
+| Paris    | Venus   | 2 To       | `200`    | `2a638e`     | `par1`       |
+| London   | Earth   | 2 To       | `200`    | `68143d`     | `lon1`       |
+| Brussels | Mars    | 1.5 To     | `150`    | `212f75`     | `bru1`       |
+
+### Identifier
+
+After its first launch, garage generates a random and unique identifier for each nodes, such as:
+
+```
+8781c50c410a41b363167e9d49cc468b6b9e4449b6577b64f15a249a149bdcbc
+```
+
+Often a shorter form can be used, containing only the beginning of the identifier, like `8781c5`,
+which identifies the server "Mercury" located in "Paris" according to our previous table.
+
+The most simple way to match an identifier to a node is to run:
+
+```
+garagectl status
+```
+
+It will display the IP address associated with each node; from the IP address you will be able to recognize the node.
+
+### Tokens
+
+Garage reasons on an arbitrary metric about disk storage that is named "tokens".
+The number of tokens must be proportional to the disk space dedicated to the node.
+Additionaly, ideally the number of tokens must be in the order of magnitude of 100
+to provide a good trade-off between data load balancing and performances (*this sentence must be verified, it may be wrong*).
+
+Here we chose 1 token = 10 Go but you are free to select the value that best fit your needs.
+
+### Datacenter
+
+Datacenter are simply a user-chosen identifier that identify a group of server that are located in the same place.
+It is up to the system administrator deploying garage to identify what does "the same place" means.
+Behind the scene, garage will try to store the same data on different sites to provide high availability despite a data center failure.
+
+### Inject the topology
+
+Given the information above, we will configure our cluster as follow:
+
+```
+garagectl node configure --datacenter par1 -n 100 -t mercury 8781c5
+garagectl node configure --datacenter par1 -n 200 -t venus 2a638e
+garagectl node configure --datacenter lon1 -n 200 -t earth 68143d
+garagectl node configure --datacenter bru1 -n 150 -t mars 212f75
+```
diff --git a/doc/book/src/getting_started/control.md b/doc/book/src/getting_started/control.md
new file mode 100644
index 00000000..9a66a0dc
--- /dev/null
+++ b/doc/book/src/getting_started/control.md
@@ -0,0 +1,77 @@
+# Control the daemon
+
+The `garage` binary has two purposes:
+  - it acts as a daemon when launched with `garage server ...`
+  - it acts as a control tool for the daemon when launched with any other command
+
+In this section, we will see how to use the `garage` binary as a control tool for the daemon we just started.
+You first need to get a shell having access to this binary, which depends of your configuration:
+  - with `docker-compose`, run `sudo docker-compose exec g1 bash` then `/garage/garage`
+  - with `docker`, run `sudo docker exec -ti garaged bash` then `/garage/garage`
+  - with `systemd`, simply run `/usr/local/bin/garage` if you followed previous instructions
+
+*You can also install the binary on your machine to remotely control the cluster.*
+
+## Talk to the daemon and create an alias
+
+`garage` requires 4 options to talk with the daemon:
+
+```
+--ca-cert <ca-cert>            
+--client-cert <client-cert>    
+--client-key <client-key>      
+-h, --rpc-host <rpc-host>
+```
+
+The 3 first ones are certificates and keys needed by TLS, the last one is simply the address of garage's RPC endpoint.
+Because we configure garage directly from the server, we do not need to set `--rpc-host`.
+To avoid typing the 3 first options each time we want to run a command, we will create an alias.
+
+### `docker-compose` alias
+
+```bash
+alias garagectl='/garage/garage \
+  --ca-cert /pki/garage-ca.crt \
+  --client-cert /pki/garage.crt \
+  --client-key /pki/garage.key'
+```
+
+### `docker` alias
+
+```bash
+alias garagectl='/garage/garage \
+  --ca-cert /etc/garage/pki/garage-ca.crt \
+  --client-cert /etc/garage/pki/garage.crt \
+  --client-key /etc/garage/pki/garage.key'
+```
+
+
+### raw binary alias
+
+```bash
+alias garagectl='/usr/local/bin/garage \
+  --ca-cert /etc/garage/pki/garage-ca.crt \
+  --client-cert /etc/garage/pki/garage.crt \
+  --client-key /etc/garage/pki/garage.key'
+```
+
+Of course, if your deployment does not match exactly one of this alias, feel free to adapt it to your needs!
+
+## Test the alias
+
+You can test your alias by running a simple command such as:
+
+```
+garagectl status
+```
+
+You should get something like that as result:
+
+```
+Healthy nodes:
+2a638ed6c775b69a…	37f0ba978d27	[::ffff:172.20.0.101]:3901	UNCONFIGURED/REMOVED
+68143d720f20c89d…	9795a2f7abb5	[::ffff:172.20.0.103]:3901	UNCONFIGURED/REMOVED
+8781c50c410a41b3…	758338dde686	[::ffff:172.20.0.102]:3901	UNCONFIGURED/REMOVED
+```
+
+...which means that you are ready to configure your cluster!
diff --git a/doc/book/src/getting_started/daemon.md b/doc/book/src/getting_started/daemon.md
new file mode 100644
index 00000000..2f2b71b0
--- /dev/null
+++ b/doc/book/src/getting_started/daemon.md
@@ -0,0 +1,222 @@
+# Configure the daemon
+
+Garage is a software that can be run only in a cluster and requires at least 3 instances.
+In our getting started guide, we document two deployment types:
+  - [Test deployment](#test-deployment) though `docker-compose`
+  - [Real-world deployment](#real-world-deployment) through `docker` or `systemd`
+
+In any case, you first need to generate TLS certificates, as traffic is encrypted between Garage's nodes.
+
+## Generating a TLS Certificate
+
+To generate your TLS certificates, run on your machine:
+
+```
+wget https://git.deuxfleurs.fr/Deuxfleurs/garage/raw/branch/master/genkeys.sh
+chmod +x genkeys.sh
+./genkeys.sh
+```
+
+It will creates a folder named `pki` containing the keys that you will used for the cluster.
+
+## Test deployment
+
+Single machine deployment is only described through `docker-compose`.
+
+Before starting, we recommend you create a folder for our deployment:
+
+```bash
+mkdir garage-single
+cd garage-single
+```
+
+We start by creating a file named `docker-compose.yml` describing our network and our containers:
+
+```yml
+version: '3.4'
+
+networks: { virtnet: { ipam: { config: [ subnet: 172.20.0.0/24 ]}}}
+
+services:
+  g1:
+    image: lxpz/garage_amd64:v0.1.1d
+    networks: { virtnet: { ipv4_address: 172.20.0.101 }}
+    volumes:
+      - "./pki:/pki"
+      - "./config.toml:/garage/config.toml"
+
+  g2:
+    image: lxpz/garage_amd64:v0.1.1d
+    networks: { virtnet: { ipv4_address: 172.20.0.102 }}
+    volumes:
+      - "./pki:/pki"
+      - "./config.toml:/garage/config.toml"
+
+  g3:
+    image: lxpz/garage_amd64:v0.1.1d
+    networks: { virtnet: { ipv4_address: 172.20.0.103 }}
+    volumes:
+      - "./pki:/pki"
+      - "./config.toml:/garage/config.toml"
+```
+
+*We define a static network here which is not considered as a best practise on Docker.
+The rational is that Garage only supports IP address and not domain names in its configuration, so we need to know the IP address in advance.*
+
+and then create the `config.toml` file next to it as follow:
+
+```toml
+metadata_dir = "/garage/meta"
+data_dir = "/garage/data"
+rpc_bind_addr = "[::]:3901"
+bootstrap_peers = [
+  "172.20.0.101:3901",
+  "172.20.0.102:3901",
+  "172.20.0.103:3901",
+]
+
+[rpc_tls]
+ca_cert = "/pki/garage-ca.crt"
+node_cert = "/pki/garage.crt"
+node_key = "/pki/garage.key"
+
+[s3_api]
+s3_region = "garage"
+api_bind_addr = "[::]:3900"
+
+[s3_web]
+bind_addr = "[::]:3902"
+root_domain = ".web.garage"
+index = "index.html"
+```
+
+*Please note that we have not mounted `/garage/meta` or `/garage/data` on the host: data will be lost when the container will be destroyed.*
+
+And that's all, you are ready to launch your cluster!
+
+```
+sudo docker-compose up
+```
+
+While your daemons are up, your cluster is still not configured yet.
+However, you can check that your services are still listening as expected by querying them from your host:
+
+```bash
+curl http://172.20.0.{101,102,103}:3902
+```
+
+which should give you:
+
+```
+Not found
+Not found
+Not found
+```
+
+That's all, you are ready to [configure your cluster!](./cluster.md).
+
+## Real-world deployment
+
+Before deploying garage on your infrastructure, you must inventory your machines.
+For our example, we will suppose the following infrastructure:
+
+| Location | Name    | IP Address | Disk Space |
+|----------|---------|------------|------------|
+| Paris    | Mercury | fc00:1::1  | 1 To       |
+| Paris    | Venus   | fc00:1::2  | 2 To       |
+| London   | Earth   | fc00:1::2  | 2 To       |
+| Brussels | Mars    | fc00:B::1  | 1.5 To     |
+
+On each machine, we will have a similar setup, especially you must consider the following folders/files:
+  - `/etc/garage/pki`: Garage certificates, must be generated on your computer and copied on the servers
+  - `/etc/garage/config.toml`: Garage daemon's configuration (defined below)
+  - `/etc/systemd/system/garage.service`: Service file to start garage at boot automatically (defined below, not required if you use docker)
+  - `/var/lib/garage/meta`: Contains Garage's metadata, put this folder on a SSD if possible
+  - `/var/lib/garage/data`: Contains Garage's data, this folder will grows and must be on a large storage, possibly big HDDs.
+
+A valid `/etc/garage/config.toml` for our cluster would be:
+
+```toml
+metadata_dir = "/var/lib/garage/meta"
+data_dir = "/var/lib/garage/data"
+rpc_bind_addr = "[::]:3901"
+bootstrap_peers = [
+  "[fc00:1::1]:3901",
+  "[fc00:1::2]:3901",
+  "[fc00:B::1]:3901",
+  "[fc00:F::1]:3901",
+]
+
+[rpc_tls]
+ca_cert = "/etc/garage/pki/garage-ca.crt"
+node_cert = "/etc/garage/pki/garage.crt"
+node_key = "/etc/garage/pki/garage.key"
+
+[s3_api]
+s3_region = "garage"
+api_bind_addr = "[::]:3900"
+
+[s3_web]
+bind_addr = "[::]:3902"
+root_domain = ".web.garage"
+index = "index.html"
+```
+
+Please make sure to change `bootstrap_peers` to **your** IP addresses!
+
+### For docker users
+
+On each machine, you can run the daemon with:
+
+```bash
+docker run \
+  -d \
+  --name garaged \
+  --restart always \
+  --network host \
+  -v /etc/garage/pki:/etc/garage/pki \
+  -v /etc/garage/config.toml:/garage/config.toml \
+  -v /var/lib/garage/meta:/var/lib/garage/meta \
+  -v /var/lib/garage/data:/var/lib/garage/data \
+  lxpz/garage_amd64:v0.1.1d
+```
+
+It should be restart automatically at each reboot.
+Please note that we use host networking as otherwise Docker containers can no communicate with IPv6.
+
+To upgrade, simply stop and remove this container and start again the command with a new version of garage.
+
+### For systemd/raw binary users
+
+Create a file named `/etc/systemd/system/garage.service`:
+
+```toml
+[Unit]
+Description=Garage Data Store
+After=network-online.target
+Wants=network-online.target
+
+[Service]
+Environment='RUST_LOG=garage=info' 'RUST_BACKTRACE=1'
+ExecStart=/usr/local/bin/garage server -c /etc/garage/config.toml
+
+[Install]
+WantedBy=multi-user.target
+```
+
+To start the service then automatically enable it at boot:
+
+```bash
+sudo systemctl start garage
+sudo systemctl enable garage
+```
+
+To see if the service is running and to browse its logs:
+
+```bash
+sudo systemctl status garage
+sudo journalctl -u garage
+```
+
+If you want to modify the service file, do not forget to run `systemctl daemon-reload`
+to inform `systemd` of your modifications.
diff --git a/doc/book/src/getting_started/files.md b/doc/book/src/getting_started/files.md
new file mode 100644
index 00000000..0e3939ce
--- /dev/null
+++ b/doc/book/src/getting_started/files.md
@@ -0,0 +1,42 @@
+# Handle files
+
+We recommend the use of MinIO Client to interact with Garage files (`mc`).
+Instructions to install it and use it are provided on the [MinIO website](https://docs.min.io/docs/minio-client-quickstart-guide.html).
+Before reading the following, you need a working `mc` command on your path.
+
+## Configure `mc`
+
+You need your access key and secret key created in the [previous section](bucket.md).
+You also need to set the endpoint: it must match the IP address of one of the node of the cluster and the API port (3900 by default).
+For this whole configuration, you must set an alias name: we chose `my-garage`, that you will used for all commands.
+
+Adapt the following command accordingly and run it:
+
+```bash
+mc alias set \
+  my-garage \
+  http://172.20.0.101:3900 \
+  <access key> \
+  <secret key> \
+  --api S3v4
+```
+
+You must also add an environment variable to your configuration to inform MinIO of our region (`garage` by default).
+The best way is to add the following snippet to your `$HOME/.bash_profile` or `$HOME/.bashrc` file:
+
+```bash
+export MC_REGION=garage
+```
+
+## Use `mc`
+
+You can not list buckets from `mc` currently.
+
+But the following commands and many more should work:
+
+```bash
+mc cp image.png my-garage/nextcloud-bucket
+mc cp my-garage/nextcloud-bucket/image.png .
+mc ls my-garage/nextcloud-bucket
+mc mirror localdir/ my-garage/another-bucket
+```
diff --git a/doc/book/src/getting_started/index.md b/doc/book/src/getting_started/index.md
new file mode 100644
index 00000000..282f5034
--- /dev/null
+++ b/doc/book/src/getting_started/index.md
@@ -0,0 +1,5 @@
+# Getting Started
+
+Let's start your Garage journey!
+In this chapter, we explain how to deploy a simple garage cluster and start interacting with it.
+Our goal is to introduce you to Garage's workflows.