diff options
Diffstat (limited to 'doc/book/src/development/miscellaneous_notes.md')
| -rw-r--r-- | doc/book/src/development/miscellaneous_notes.md | 134 |
1 files changed, 134 insertions, 0 deletions
diff --git a/doc/book/src/development/miscellaneous_notes.md b/doc/book/src/development/miscellaneous_notes.md new file mode 100644 index 00000000..c259cba1 --- /dev/null +++ b/doc/book/src/development/miscellaneous_notes.md @@ -0,0 +1,134 @@ +# Miscellaneous Notes + +## Quirks about cargo2nix/rust in Nix + +If you use submodules in your crate (like `crdt` and `replication` in `garage_table`), you must list them in `default.nix` + +The Windows target does not work. it might be solvable through [overrides](https://github.com/cargo2nix/cargo2nix/blob/master/overlay/overrides.nix). Indeed, we pass `x86_64-pc-windows-gnu` but mingw need `x86_64-w64-mingw32` + +We have a simple [PR on cargo2nix](https://github.com/cargo2nix/cargo2nix/pull/201) that fixes critical bugs but the project does not seem very active currently. We must use [my patched version of cargo2nix](https://github.com/superboum/cargo2nix) to enable i686 and armv6l compilation. We might need to contribute to cargo2nix in the future. + + +## Nix + +Nix has no armv7 + musl toolchains but armv7l is backward compatible with armv6l. + +Signing keys are generated with: + +``` +nix-store --generate-binary-cache-key nix.web.deuxfleurs.fr cache-priv-key.pem cache-pub-key.pem +``` + +We copy the secret key in our nix folder: + +``` +cp cache-priv-key.pem /etc/nix/signing-key.sec +``` + +Manually sign + +We can sign the whole store with: + +``` +nix sign-paths --all -k /etc/nix/signing-key.sec +``` + +Or simply the current package and its dependencies with: + +``` +nix sign-paths --recursive -k /etc/nix/signing-key.sec +``` + +Setting a key in `nix.conf` will do the signature at build time automatically without additional commands, edit the `nix.conf` of your builder: + +```toml +secret-key-files = /etc/nix/signing-key.sec +max-jobs = auto +cores = 8 +``` + +Now you are ready to build your packages: + +```bash +cat > $HOME/.awsrc <<EOF +export AWS_ACCESS_KEY_ID="xxx" +export AWS_SECRET_ACCESS_KEY="xxx" +EOF + +# source each time you want to send on the cache +source ~/.awsrc + +# copy garage build dependencies (and not only the output) +nix-build +nix-store -qR --include-outputs $(nix-instantiate default.nix) + | xargs nix copy --to 's3://nix?endpoint=garage.deuxfleurs.fr®ion=garage' + +# copy shell dependencies +nix-build shell.nix -A inputDerivation +nix copy $(nix-store -qR result/) --to 's3://nix?endpoint=garage.deuxfleurs.fr®ion=garage' +``` + +More example of nix-copy + +``` +# nix-build produces a result/ symlink +nix copy result/ --to 's3://nix?endpoint=garage.deuxfleurs.fr®ion=garage' + +# alternative ways to use nix copy +nix copy nixpkgs.garage --to ... +nix copy /nix/store/3rbb9qsc2w6xl5xccz5ncfhy33nzv3dp-crate-garage-0.3.0 --to ... +``` + + +Clear the cache: + +```bash +mc rm --recursive --force garage/nix/ +``` + +--- + +A desirable `nix.conf` for a consumer: + +```toml +substituters = https://cache.nixos.org https://nix.web.deuxfleurs.fr +trusted-public-keys = cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY= nix.web.deuxfleurs.fr:eTGL6kvaQn6cDR/F9lDYUIP9nCVR/kkshYfLDJf1yKs= +``` + +And now, whenever you run a command like: + +``` +nix-shell +nix-build +``` + +Our cache will be checked. + +### Some references about Nix + + + - https://doc.rust-lang.org/nightly/rustc/platform-support.html + - https://nix.dev/tutorials/cross-compilation + - https://nixos.org/manual/nix/unstable/package-management/s3-substituter.html + - https://fzakaria.com/2020/09/28/nix-copy-closure-your-nix-shell.html + - http://www.lpenz.org/articles/nixchannel/index.html + + +## Drone + +Do not try to set a build as trusted from the interface or the CLI tool, +your request would be ignored. Instead, directly edit the database (table `repos`, column `repo_trusted`). + +Drone can do parallelism both at the step and the pipeline level. At the step level, parallelism is restricted to the same runner. + +## Building Docker containers + +We were: + - Unable to use the official Docker plugin because + - it requires to mount docker socket in the container but it is not recommended + - you cant set the platform when building + - Unable to use buildah because it needs `CLONE_USERNS` capability + - Unable to use the kaniko plugin for Drone as we can't set the target platform + - Unable to use the kaniko container provided by Google as we can't run arbitrary logic: we need to put our secret in .docker/config.json. + +Finally we chose to build kaniko through nix and use it in a `nix-shell`. |