Commit message (Collapse) | Author | Age | Files | Lines | ||
---|---|---|---|---|---|---|
... | ||||||
| * | improvements in block manager | Alex Auvolat | 2022-09-12 | 1 | -13/+16 | |
| | | ||||||
| * | Merge branch 'lx-perf-improvements' into netapp-stream-body | Alex Auvolat | 2022-09-08 | 42 | -1068/+1332 | |
| |\ | ||||||
| * | | Faster copy, better get error message | Alex Auvolat | 2022-09-06 | 2 | -9/+7 | |
| | | | ||||||
| * | | Merge branch 'lx-perf-improvements' into netapp-stream-body | Alex Auvolat | 2022-09-06 | 11 | -535/+735 | |
| |\ \ | ||||||
| * | | | Update netapp to lastest git version with LAS scheduling | Alex Auvolat | 2022-09-06 | 2 | -65/+32 | |
| | | | | ||||||
| * | | | Reenable node ordering | Alex Auvolat | 2022-09-06 | 1 | -6/+4 | |
| | | | | ||||||
| * | | | Make use of BytesBuf from new Netapp | Alex Auvolat | 2022-09-02 | 1 | -33/+10 | |
| | | | | ||||||
| * | | | cargo fmt | Alex Auvolat | 2022-09-02 | 1 | -1/+2 | |
| | | | | ||||||
| * | | | Apply PRIO_SECONDARY to block data transfers | Alex Auvolat | 2022-09-01 | 2 | -4/+4 | |
| | | | | ||||||
| * | | | update cargo.nix | Alex Auvolat | 2022-09-01 | 1 | -5/+5 | |
| | | | | ||||||
| * | | | Less strict timeouts | Alex Auvolat | 2022-09-01 | 7 | -10/+16 | |
| | | | | ||||||
| * | | | update netapp: straming + fix-ping | Alex Auvolat | 2022-09-01 | 2 | -10/+9 | |
| | | | | ||||||
| * | | | Update to Netapp with OrderTag support and exploit OrderTags | Alex Auvolat | 2022-09-01 | 5 | -26/+64 | |
| | | | | ||||||
| * | | | netapp recv with unbounded channel removes deadlock | Alex Auvolat | 2022-09-01 | 2 | -6/+6 | |
| | | | | ||||||
| * | | | Fix bytes_read counter | Alex Auvolat | 2022-08-31 | 1 | -4/+9 | |
| | | | | ||||||
| * | | | update netapp git commit | Alex Auvolat | 2022-08-31 | 2 | -7/+7 | |
| | | | | ||||||
| * | | | Merge branch 'lx-perf-improvements' into netapp-stream-body | Alex Auvolat | 2022-08-31 | 4 | -8/+214 | |
| |\ \ \ | ||||||
| * | | | | Try to fix clippy | Alex Auvolat | 2022-08-29 | 1 | -1/+1 | |
| | | | | | ||||||
| * | | | | cargo2nix fix to fetchCrateGit | Alex Auvolat | 2022-08-29 | 1 | -4/+4 | |
| | | | | | ||||||
| * | | | | Merge branch 'lx-perf-improvements' into netapp-stream-body | Alex Auvolat | 2022-08-29 | 1 | -1/+1 | |
| |\ \ \ \ | ||||||
| * \ \ \ \ | Merge branch 'lx-perf-improvements' into netapp-stream-body | Alex Auvolat | 2022-08-29 | 13 | -83/+116 | |
| |\ \ \ \ \ | ||||||
| * | | | | | | Factor out node request order selection logic & use in manager | Alex Auvolat | 2022-07-29 | 4 | -41/+60 | |
| | | | | | | | ||||||
| * | | | | | | Update cargo.nix | Alex Auvolat | 2022-07-29 | 1 | -8/+8 | |
| | | | | | | | ||||||
| * | | | | | | update netapp | Alex Auvolat | 2022-07-29 | 1 | -1/+1 | |
| | | | | | | | ||||||
| * | | | | | | Update cargo.nix | Alex Auvolat | 2022-07-29 | 1 | -73/+103 | |
| | | | | | | | ||||||
| * | | | | | | Fix clippy | Alex Auvolat | 2022-07-29 | 2 | -5/+1 | |
| | | | | | | | ||||||
| * | | | | | | Use streaming in block manager | Alex Auvolat | 2022-07-29 | 7 | -88/+284 | |
| | | | | | | | ||||||
| * | | | | | | update netapp to 0.5 | Alex Auvolat | 2022-07-29 | 5 | -10/+10 | |
| | | | | | | | ||||||
| * | | | | | | First adaptation to WIP netapp with streaming body | Alex Auvolat | 2022-07-29 | 12 | -112/+97 | |
| | | | | | | | ||||||
* | | | | | | | Merge pull request 'performance improvements' (#342) from ↵ | Alex | 2022-09-12 | 16 | -310/+646 | |
|\ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | lx-perf-improvements into main Performance improvements included in this PR: - [x] Use `Bytes` at a few places where appropriate, instead of `Vec<u8>`, to reduce the number of copies - [x] StreamChunker now accumulates incoming slices in a `Vec<Bytes>` instead of a `VecDeque<u8>`. Replaces calls to `.extend()` and `.drain()` that were quite costly by a simple `concat()` on a vec of slices which is much more optimized - [x] Hashing (b2, sha256, md5) is now done on a Tokio thread dedicated to cpu-intensive tasks, using `spawn_blocking` - [x] Block manager now uses 256 independant locks instead of one big lock for writing, reduces contention when writing several/many objects in parallel - [x] Better LMDB defaults: we now put flags `NoSync` and `NoMetaSync` to avoid `fsync` at each transaction (extremely slow). Also increased number of LMDB readers to accomodate more intensive workloads Other changes included in this PR: - [x] Update to hashing and MAC crates: md5 and sha2 from 0.9 to 0.10, hmac from 0.10 to 0.12 - [x] switch to `tracing_subscriber` for logs, which allows to have timing of each event Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/342 | |||||
| * | | | | | | | Simplify+improve async hasher by using bounded channellx-perf-improvements | Alex Auvolat | 2022-09-12 | 1 | -8/+5 | |
| | |_|_|_|_|/ | |/| | | | | | ||||||
| * | | | | | | Merge branch 'main' into lx-perf-improvements | Alex Auvolat | 2022-09-08 | 42 | -1076/+1341 | |
| |\ \ \ \ \ \ | |/ / / / / / |/| | | | | / | | |_|_|_|/ | |/| | | | | ||||||
* | | | | | | Merge pull request 'Reorganize dependencies' (#373) from improve-deps into main | Alex | 2022-09-08 | 43 | -943/+1288 | |
|\ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This PR includes work from @jirutka : - [x] Allow linking against system-provided libraries (libsodium, libsqlite, libzstd) #370 - [x] Make OTLP exporter optional and allow building without Prometheus exporter (/metrics) #372 And also: - [x] Update `.nix` files - [x] Remove heed default-features - [x] Bump versions of all Garage crates to 0.8.0 - [x] Make db engines (lmdb, sled, sqlite) optionnal - [x] Add documentation for available features - [x] Directly include code of previous versions used for migration in order to reduce dependencies - [x] Read variable `GIT_VERSION` from garage main instead of in crate garage_util to make builds faster - [x] Report features used in the build somewhere? (in `garage --version` or something) - [x] Check we `warn!` correctly if we try to use deactivated feature - [x] Allow not to launch S3 endpoint if not in config Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/373 | |||||
| * | | | | | | Move version back into utilimprove-deps | Alex Auvolat | 2022-09-07 | 11 | -18/+21 | |
| | | | | | | | ||||||
| * | | | | | | Inject GIT_VERSION even later | Alex Auvolat | 2022-09-07 | 3 | -18/+28 | |
| | | | | | | | ||||||
| * | | | | | | Fix merge | Alex Auvolat | 2022-09-07 | 2 | -5/+5 | |
| | | | | | | | ||||||
| * | | | | | | Merge branch 'main' into improve-deps | Alex Auvolat | 2022-09-07 | 9 | -528/+725 | |
| |\ \ \ \ \ \ | |/ / / / / / |/| | | | | | | ||||||
| * | | | | | | Fix build error | Alex Auvolat | 2022-09-07 | 1 | -1/+1 | |
| | | | | | | | ||||||
| * | | | | | | Add warnings when features are not included in build | Alex Auvolat | 2022-09-07 | 2 | -0/+8 | |
| | | | | | | | ||||||
| * | | | | | | Error messages when system-libs XOR bundled-libs != 1 | Alex Auvolat | 2022-09-07 | 1 | -0/+6 | |
| | | | | | | | ||||||
| * | | | | | | Make all HTTP services optionnal | Alex Auvolat | 2022-09-07 | 7 | -275/+301 | |
| | | | | | | | ||||||
| * | | | | | | Report build features in garage --help | Alex Auvolat | 2022-09-07 | 7 | -9/+63 | |
| | | | | | | | ||||||
| * | | | | | | Move GIT_VERSION injection later in build chain to reduce build times | Alex Auvolat | 2022-09-07 | 13 | -79/+76 | |
| | | | | | | | ||||||
| * | | | | | | cargo fmt | Alex Auvolat | 2022-09-06 | 2 | -2/+0 | |
| | | | | | | | ||||||
| * | | | | | | Include code from v0.5.1 directly to remove dependencies | Alex Auvolat | 2022-09-06 | 14 | -290/+386 | |
| | | | | | | | ||||||
| * | | | | | | Disable k2v tests when feature is disabled | Alex Auvolat | 2022-09-06 | 1 | -0/+1 | |
| | | | | | | | ||||||
| * | | | | | | Remove opentelemetry-otlp dep in api/ | Alex Auvolat | 2022-09-06 | 4 | -81/+75 | |
| | | | | | | | ||||||
| * | | | | | | Update Nix files with optional db engines | Alex Auvolat | 2022-09-06 | 2 | -32/+42 | |
| | | | | | | | ||||||
| * | | | | | | Document available build features | Alex Auvolat | 2022-09-06 | 1 | -16/+33 | |
| | | | | | | | ||||||
| * | | | | | | Make all DB engines optional build features | Alex Auvolat | 2022-09-06 | 5 | -7/+50 | |
| | | | | | | |