1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
|
+++
title = "restic"
description = "restic"
weight = 10
+++
Add the admin account as `deuxfleurs` to your `~/.mc/config` file
You need to choose some names/identifiers:
```bash
export ENDPOINT="https://s3.garage.tld"
export SERVICE_NAME="example"
export BUCKET_NAME="backups-${SERVICE_NAME}"
export NEW_ACCESS_KEY_ID="key-${SERVICE_NAME}"
export NEW_SECRET_ACCESS_KEY=$(openssl rand -base64 32)
export POLICY_NAME="policy-$BUCKET_NAME"
```
Create a new bucket:
```bash
mc mb deuxfleurs/$BUCKET_NAME
```
Create a new user:
```bash
mc admin user add deuxfleurs $NEW_ACCESS_KEY_ID $NEW_SECRET_ACCESS_KEY
```
Add this new user to your `~/.mc/config.json`, run this command before to generate the snippet to copy/paste:
```
cat > /dev/stdout <<EOF
"$NEW_ACCESS_KEY_ID": {
"url": "$ENDPOINT",
"accessKey": "$NEW_ACCESS_KEY_ID",
"secretKey": "$NEW_SECRET_ACCESS_KEY",
"api": "S3v4",
"path": "auto"
},
EOF
```
---
Create a policy for this bucket and save it as json:
```bash
cat > /tmp/policy.json <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::${BUCKET_NAME}"
]
},
{
"Effect": "Allow",
"Action": [
"s3:*"
],
"Resource": [
"arn:aws:s3:::${BUCKET_NAME}/*"
]
}
]
}
EOF
```
Register it:
```bash
mc admin policy add deuxfleurs $POLICY_NAME /tmp/policy.json
```
Set it to your user:
```bash
mc admin policy set deuxfleurs $POLICY_NAME user=${NEW_ACCESS_KEY_ID}
```
Now it should display *only* your new bucket when running:
```bash
mc ls $NEW_ACCESS_KEY_ID
```
---
Now we need to initialize the repository with restic.
```bash
export AWS_ACCESS_KEY_ID=$NEW_ACCESS_KEY_ID
export AWS_SECRET_ACCESS_KEY=$NEW_SECRET_ACCESS_KEY
export RESTIC_REPOSITORY="s3:$ENDPOINT/$BUCKET_NAME"
export RESTIC_PASSWORD=$(openssl rand -base64 32)
```
Then init the repo for restic from your machine:
```
restic init
```
*I am using restic version `restic 0.12.1 compiled with go1.16.9 on linux/amd64`*
See your snapshots with:
```
restic snapshots
```
Check also these useful commands:
```
restic ls
restic diff
restic help
```
---
Add the secrets to Consul, near your service secrets.
The idea is that the backuping service is a component of the global running service.
You must run in `app/<name>/secrets/<subpath>`:
```bash
echo "USER Backup AWS access key ID" > backup_aws_access_key_id
echo "USER Backup AWS secret access key" > backup_aws_secret_access_key
echo "USER Restic repository, eg. s3:https://s3.garage.tld" > backup_restic_repository
echo "USER Restic password to encrypt backups" > backup_restic_password
```
Then run secretmgr:
```bash
# Spawning a nix shell is an easy way to get all the dependencies you need
nix-shell
# Check that secretmgr works for you
python3 secretmgr.py check <name>
# Now interactively feed the secrets
python3 secretmgr.py gen <name>
```
---
Now we need a service that runs:
```
restic backup .
```
And also that garbage collect snapshots.
I propose:
```
restic forget --prune --keep-within 1m1d --keep-within-weekly 3m --keep-within-monthly 1y
```
Also try to restore a snapshot:
```
restic restore <snapshot id> --target /tmp/$SERVICE_NAME
```
|
