blob: c259cba10325f65f193cc564c013f1f2d696ee98 (
plain) (
blame)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
|
# Miscellaneous Notes
## Quirks about cargo2nix/rust in Nix
If you use submodules in your crate (like `crdt` and `replication` in `garage_table`), you must list them in `default.nix`
The Windows target does not work. it might be solvable through [overrides](https://github.com/cargo2nix/cargo2nix/blob/master/overlay/overrides.nix). Indeed, we pass `x86_64-pc-windows-gnu` but mingw need `x86_64-w64-mingw32`
We have a simple [PR on cargo2nix](https://github.com/cargo2nix/cargo2nix/pull/201) that fixes critical bugs but the project does not seem very active currently. We must use [my patched version of cargo2nix](https://github.com/superboum/cargo2nix) to enable i686 and armv6l compilation. We might need to contribute to cargo2nix in the future.
## Nix
Nix has no armv7 + musl toolchains but armv7l is backward compatible with armv6l.
Signing keys are generated with:
```
nix-store --generate-binary-cache-key nix.web.deuxfleurs.fr cache-priv-key.pem cache-pub-key.pem
```
We copy the secret key in our nix folder:
```
cp cache-priv-key.pem /etc/nix/signing-key.sec
```
Manually sign
We can sign the whole store with:
```
nix sign-paths --all -k /etc/nix/signing-key.sec
```
Or simply the current package and its dependencies with:
```
nix sign-paths --recursive -k /etc/nix/signing-key.sec
```
Setting a key in `nix.conf` will do the signature at build time automatically without additional commands, edit the `nix.conf` of your builder:
```toml
secret-key-files = /etc/nix/signing-key.sec
max-jobs = auto
cores = 8
```
Now you are ready to build your packages:
```bash
cat > $HOME/.awsrc <<EOF
export AWS_ACCESS_KEY_ID="xxx"
export AWS_SECRET_ACCESS_KEY="xxx"
EOF
# source each time you want to send on the cache
source ~/.awsrc
# copy garage build dependencies (and not only the output)
nix-build
nix-store -qR --include-outputs $(nix-instantiate default.nix)
| xargs nix copy --to 's3://nix?endpoint=garage.deuxfleurs.fr®ion=garage'
# copy shell dependencies
nix-build shell.nix -A inputDerivation
nix copy $(nix-store -qR result/) --to 's3://nix?endpoint=garage.deuxfleurs.fr®ion=garage'
```
More example of nix-copy
```
# nix-build produces a result/ symlink
nix copy result/ --to 's3://nix?endpoint=garage.deuxfleurs.fr®ion=garage'
# alternative ways to use nix copy
nix copy nixpkgs.garage --to ...
nix copy /nix/store/3rbb9qsc2w6xl5xccz5ncfhy33nzv3dp-crate-garage-0.3.0 --to ...
```
Clear the cache:
```bash
mc rm --recursive --force garage/nix/
```
---
A desirable `nix.conf` for a consumer:
```toml
substituters = https://cache.nixos.org https://nix.web.deuxfleurs.fr
trusted-public-keys = cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY= nix.web.deuxfleurs.fr:eTGL6kvaQn6cDR/F9lDYUIP9nCVR/kkshYfLDJf1yKs=
```
And now, whenever you run a command like:
```
nix-shell
nix-build
```
Our cache will be checked.
### Some references about Nix
- https://doc.rust-lang.org/nightly/rustc/platform-support.html
- https://nix.dev/tutorials/cross-compilation
- https://nixos.org/manual/nix/unstable/package-management/s3-substituter.html
- https://fzakaria.com/2020/09/28/nix-copy-closure-your-nix-shell.html
- http://www.lpenz.org/articles/nixchannel/index.html
## Drone
Do not try to set a build as trusted from the interface or the CLI tool,
your request would be ignored. Instead, directly edit the database (table `repos`, column `repo_trusted`).
Drone can do parallelism both at the step and the pipeline level. At the step level, parallelism is restricted to the same runner.
## Building Docker containers
We were:
- Unable to use the official Docker plugin because
- it requires to mount docker socket in the container but it is not recommended
- you cant set the platform when building
- Unable to use buildah because it needs `CLONE_USERNS` capability
- Unable to use the kaniko plugin for Drone as we can't set the target platform
- Unable to use the kaniko container provided by Google as we can't run arbitrary logic: we need to put our secret in .docker/config.json.
Finally we chose to build kaniko through nix and use it in a `nix-shell`.
|
