aboutsummaryrefslogblamecommitdiff
path: root/op_guide/restic/README.md
blob: cf67d9227ac83b8ed2081eea5d1dabb3132db1f5 (plain) (tree)
1
2
3
4
5
6
7
8




                                                                 

                                       
 


                                              
                                                       














                                                                      




                                                                                                             
                                   






                                              





























                                                    
   













                                                                     
                                                          
 
       
                        






                                                     





                                                    










                                                     













                                                                                 







                                 



                                                                                    









                                                                                      
 









                                                                          














                                                                                         


                                                                                                            
Add the admin account as `deuxfleurs` to your `~/.mc/config` file

You need to choose some names/identifiers:

```bash
export ENDPOINT="https://s3.garage.tld"
export SERVICE_NAME="example"


export BUCKET_NAME="backups-${SERVICE_NAME}"
export NEW_ACCESS_KEY_ID="key-${SERVICE_NAME}"
export NEW_SECRET_ACCESS_KEY=$(openssl rand -base64 32)
export POLICY_NAME="policy-$BUCKET_NAME"
```

Create a new bucket:

```bash
mc mb deuxfleurs/$BUCKET_NAME
```

Create a new user:

```bash
mc admin user add deuxfleurs $NEW_ACCESS_KEY_ID $NEW_SECRET_ACCESS_KEY
```

Add this new user to your `~/.mc/config.json`, run this command before to generate the snippet to copy/paste:

```
cat > /dev/stdout <<EOF
"$NEW_ACCESS_KEY_ID": {
	"url": "https://$ENDPOINT",
	"accessKey": "$NEW_ACCESS_KEY_ID",
	"secretKey": "$NEW_SECRET_ACCESS_KEY",
	"api": "S3v4",
	"path": "auto"
},
EOF
```

---

Create a policy for this bucket and save it as json:

```bash
cat > /tmp/policy.json <<EOF
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket"
            ],
            "Resource": [
                "arn:aws:s3:::${BUCKET_NAME}"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:*"
            ],
            "Resource": [
                "arn:aws:s3:::${BUCKET_NAME}/*"
            ]
        }
    ]
}
EOF
```

Register it:

```bash
mc admin policy add deuxfleurs $POLICY_NAME /tmp/policy.json
```

Set it to your user:

```bash
mc admin policy set deuxfleurs $POLICY_NAME user=${NEW_ACCESS_KEY_ID}
```

Now it should display *only* your new bucket when running:

```bash
mc ls $NEW_ACCESS_KEY_ID
```

---

Now we need to initialize the repository with restic.

```bash
export AWS_ACCESS_KEY_ID=$NEW_ACCESS_KEY_ID
export AWS_SECRET_ACCESS_KEY=$NEW_SECRET_ACCESS_KEY
export RESTIC_REPOSITORY="s3:$ENDPOINT/$BUCKET_NAME"
export RESTIC_PASSWORD=$(openssl rand -base64 32)
```

Save the password:

```bash
echo $RESTIC_PASSWORD
pass deuxfleurs/backups/$SERVICE_NAME/restic_password
# ctrl + c -> ctrl + v
cd ~/.password-store/deuxfleurs/
git pull ; git push
cd -
```

Then init the repo for restic from your machine:

```
restic init
```

*I am using restic version `restic 0.12.1 compiled with go1.16.9 on linux/amd64`*

See your snapshots with:

```
restic snapshots
```

Check also these useful commands:

```
restic ls
restic diff
restic help
```

---

Add the secrets to Consul, near your service secrets.
The idea is that the backuping service is a component of the global running service.
You must run in `app/<name>/secrets/<subpath>`:

```bash
echo "USER Backup AWS access key ID" > backup_aws_access_key_id
echo "USER Backup AWS secret access key" > backup_aws_secret_access_key
echo "USER Restic repository, eg. s3:https://s3.garage.tld" > backup_restic_repository
echo "USER Restic password to encrypt backups" > backup_restic_password
```

Then run secretmgr:

```bash
# Spawning a nix shell is an easy way to get all the dependencies you need
nix-shell

# Check that secretmgr works for you
python3 secretmgr.py check <name>

# Now interactively feed the secrets
python3 secretmgr.py gen <name>
```

---

Now we need a service that runs:

```
restic backup .
```

And also that garbage collect snapshots.
I propose:

```
restic forget --prune --keep-within 1m1d --keep-within-weekly 3m --keep-within-monthly 1y
```

Find an existing .hcl declaration that uses restic in this repository or in the Deuxfleurs/nixcfg repository
to use it as an example.