1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
|
# Configure the daemon
Garage is a software that can be run only in a cluster and requires at least 3 instances.
In our getting started guide, we document two deployment types:
- [Test deployment](#test-deployment) though `docker-compose`
- [Real-world deployment](#real-world-deployment) through `docker` or `systemd`
In any case, you first need to generate TLS certificates, as traffic is encrypted between Garage's nodes.
## Generating a TLS Certificate
To generate your TLS certificates, run on your machine:
```
wget https://git.deuxfleurs.fr/Deuxfleurs/garage/raw/branch/master/genkeys.sh
chmod +x genkeys.sh
./genkeys.sh
```
It will creates a folder named `pki` containing the keys that you will used for the cluster.
## Test deployment
Single machine deployment is only described through `docker-compose`.
Before starting, we recommend you create a folder for our deployment:
```bash
mkdir garage-single
cd garage-single
```
We start by creating a file named `docker-compose.yml` describing our network and our containers:
```yml
version: '3.4'
networks: { virtnet: { ipam: { config: [ subnet: 172.20.0.0/24 ]}}}
services:
g1:
image: lxpz/garage_amd64:v0.1.1d
networks: { virtnet: { ipv4_address: 172.20.0.101 }}
volumes:
- "./pki:/pki"
- "./config.toml:/garage/config.toml"
g2:
image: lxpz/garage_amd64:v0.1.1d
networks: { virtnet: { ipv4_address: 172.20.0.102 }}
volumes:
- "./pki:/pki"
- "./config.toml:/garage/config.toml"
g3:
image: lxpz/garage_amd64:v0.1.1d
networks: { virtnet: { ipv4_address: 172.20.0.103 }}
volumes:
- "./pki:/pki"
- "./config.toml:/garage/config.toml"
```
*We define a static network here which is not considered as a best practise on Docker.
The rational is that Garage only supports IP address and not domain names in its configuration, so we need to know the IP address in advance.*
and then create the `config.toml` file next to it as follow:
```toml
metadata_dir = "/garage/meta"
data_dir = "/garage/data"
rpc_bind_addr = "[::]:3901"
bootstrap_peers = [
"172.20.0.101:3901",
"172.20.0.102:3901",
"172.20.0.103:3901",
]
[rpc_tls]
ca_cert = "/pki/garage-ca.crt"
node_cert = "/pki/garage.crt"
node_key = "/pki/garage.key"
[s3_api]
s3_region = "garage"
api_bind_addr = "[::]:3900"
[s3_web]
bind_addr = "[::]:3902"
root_domain = ".web.garage"
index = "index.html"
```
*Please note that we have not mounted `/garage/meta` or `/garage/data` on the host: data will be lost when the container will be destroyed.*
And that's all, you are ready to launch your cluster!
```
sudo docker-compose up
```
While your daemons are up, your cluster is still not configured yet.
However, you can check that your services are still listening as expected by querying them from your host:
```bash
curl http://172.20.0.{101,102,103}:3902
```
which should give you:
```
Not found
Not found
Not found
```
That's all, you are ready to [configure your cluster!](./cluster.md).
## Real-world deployment
Before deploying garage on your infrastructure, you must inventory your machines.
For our example, we will suppose the following infrastructure:
| Location | Name | IP Address | Disk Space |
|----------|---------|------------|------------|
| Paris | Mercury | fc00:1::1 | 1 To |
| Paris | Venus | fc00:1::2 | 2 To |
| London | Earth | fc00:1::2 | 2 To |
| Brussels | Mars | fc00:B::1 | 1.5 To |
On each machine, we will have a similar setup, especially you must consider the following folders/files:
- `/etc/garage/pki`: Garage certificates, must be generated on your computer and copied on the servers
- `/etc/garage/config.toml`: Garage daemon's configuration (defined below)
- `/etc/systemd/system/garage.service`: Service file to start garage at boot automatically (defined below, not required if you use docker)
- `/var/lib/garage/meta`: Contains Garage's metadata, put this folder on a SSD if possible
- `/var/lib/garage/data`: Contains Garage's data, this folder will grows and must be on a large storage, possibly big HDDs.
A valid `/etc/garage/config.toml` for our cluster would be:
```toml
metadata_dir = "/var/lib/garage/meta"
data_dir = "/var/lib/garage/data"
rpc_bind_addr = "[::]:3901"
bootstrap_peers = [
"[fc00:1::1]:3901",
"[fc00:1::2]:3901",
"[fc00:B::1]:3901",
"[fc00:F::1]:3901",
]
[rpc_tls]
ca_cert = "/etc/garage/pki/garage-ca.crt"
node_cert = "/etc/garage/pki/garage.crt"
node_key = "/etc/garage/pki/garage.key"
[s3_api]
s3_region = "garage"
api_bind_addr = "[::]:3900"
[s3_web]
bind_addr = "[::]:3902"
root_domain = ".web.garage"
index = "index.html"
```
Please make sure to change `bootstrap_peers` to **your** IP addresses!
### For docker users
On each machine, you can run the daemon with:
```bash
docker run \
-d \
--name garaged \
--restart always \
--network host \
-v /etc/garage/pki:/etc/garage/pki \
-v /etc/garage/config.toml:/garage/config.toml \
-v /var/lib/garage/meta:/var/lib/garage/meta \
-v /var/lib/garage/data:/var/lib/garage/data \
lxpz/garage_amd64:v0.1.1d
```
It should be restart automatically at each reboot.
Please note that we use host networking as otherwise Docker containers can no communicate with IPv6.
To upgrade, simply stop and remove this container and start again the command with a new version of garage.
### For systemd/raw binary users
Create a file named `/etc/systemd/system/garage.service`:
```toml
[Unit]
Description=Garage Data Store
After=network-online.target
Wants=network-online.target
[Service]
Environment='RUST_LOG=garage=info' 'RUST_BACKTRACE=1'
ExecStart=/usr/local/bin/garage server -c /etc/garage/config.toml
[Install]
WantedBy=multi-user.target
```
To start the service then automatically enable it at boot:
```bash
sudo systemctl start garage
sudo systemctl enable garage
```
To see if the service is running and to browse its logs:
```bash
sudo systemctl status garage
sudo journalctl -u garage
```
If you want to modify the service file, do not forget to run `systemctl daemon-reload`
to inform `systemd` of your modifications.
|
