From d574b395ffb0c5b4e672dfef3b47971b4ec82470 Mon Sep 17 00:00:00 2001 From: Quentin Dufour Date: Mon, 26 Sep 2022 11:12:46 +0200 Subject: Skeleton of our perf article --- content/blog/2022-perf/1million.png | Bin 0 -> 250100 bytes content/blog/2022-perf/amplification.png | Bin 0 -> 145685 bytes content/blog/2022-perf/complexity.png | Bin 0 -> 198406 bytes content/blog/2022-perf/db_engine.png | Bin 0 -> 170092 bytes content/blog/2022-perf/index.md | 75 +++++++++++++++++++++++++++++++ content/blog/2022-perf/io.png | Bin 0 -> 193869 bytes content/blog/2022-perf/ttfb.png | Bin 0 -> 131548 bytes 7 files changed, 75 insertions(+) create mode 100644 content/blog/2022-perf/1million.png create mode 100644 content/blog/2022-perf/amplification.png create mode 100644 content/blog/2022-perf/complexity.png create mode 100644 content/blog/2022-perf/db_engine.png create mode 100644 content/blog/2022-perf/index.md create mode 100644 content/blog/2022-perf/io.png create mode 100644 content/blog/2022-perf/ttfb.png diff --git a/content/blog/2022-perf/1million.png b/content/blog/2022-perf/1million.png new file mode 100644 index 0000000..c7ca528 Binary files /dev/null and b/content/blog/2022-perf/1million.png differ diff --git a/content/blog/2022-perf/amplification.png b/content/blog/2022-perf/amplification.png new file mode 100644 index 0000000..8962b65 Binary files /dev/null and b/content/blog/2022-perf/amplification.png differ diff --git a/content/blog/2022-perf/complexity.png b/content/blog/2022-perf/complexity.png new file mode 100644 index 0000000..a5cf631 Binary files /dev/null and b/content/blog/2022-perf/complexity.png differ diff --git a/content/blog/2022-perf/db_engine.png b/content/blog/2022-perf/db_engine.png new file mode 100644 index 0000000..0f22d6d Binary files /dev/null and b/content/blog/2022-perf/db_engine.png differ diff --git a/content/blog/2022-perf/index.md b/content/blog/2022-perf/index.md new file mode 100644 index 0000000..4abcf8d --- /dev/null +++ b/content/blog/2022-perf/index.md @@ -0,0 +1,75 @@ ++++ +title="Bringing theoretical design and real word performances face to face" +date=2022-09-26 ++++ + + +*We* + + + +--- + +## ⚠️ Disclaimer +The following results must be taken with a critical grain of salt due to some limitations that are inherent to any benchmark. We try to reference them in the following. + +Most of our tests are done on simulated networks that can not represent all the diversity of real networks (dynamic drop, jitter, latency, all of them could possibly be correlated with throughput or any other external event). We also limited ourselves to very small workloads that are not representative of a production cluster. + +For some benchmarks, we used Minio as a reference. It must be noted that we did not try to optimize its configuration as we have done on Garage, and more generally, we have way less knowledge on Minio than on Garage. +It must also be noted that Gare and Minio are systems with different feature set, *eg.* Minio supports erasure coding for better data density while Garage doesn't. + +Impact of the testing environment is also not evaluated (kernel patches, configuration, parameters, filesystem, etc.), some of these configurations could favor one configuration/software over another. Finally, our results are also provided without statistical tests to check their significance, and thus might be statistically not significative. + +In any case, we are not making a business or technical recommendation, we only share bits of our development process. +Read [benchmarking crimes](https://gernot-heiser.org/benchmarking-crimes.html) and make your own tests if you need to take a decision! + +## About our testing environment + +- Grid 5k. + +- My own computer. + +## Efficient I/O + +- streaming + +![](ttfb.png) + +- fsync, semaphore, timeouts, etc. + +![](io.png) + +## Million of objects + +- metadata engine + +![](db_engine.png) + +- storing metadata at scale + +![](1million.png) + +## Topology versatility + +- low bandwidth + +![]() + +- high network latency. phenomenon we name amplification + +![](amplification.png) + +- complexity (constant time) + +![](complexity.png) + + +## Future work + +- srpt + +- better analysis of the fsync / data reliability impact + +- analysis and comparison of Garage at scale + +- try to better understand ecosystem (riak cs, minio, ceph, swift) -> some knowledge to get diff --git a/content/blog/2022-perf/io.png b/content/blog/2022-perf/io.png new file mode 100644 index 0000000..f581a22 Binary files /dev/null and b/content/blog/2022-perf/io.png differ diff --git a/content/blog/2022-perf/ttfb.png b/content/blog/2022-perf/ttfb.png new file mode 100644 index 0000000..c0335bd Binary files /dev/null and b/content/blog/2022-perf/ttfb.png differ -- cgit v1.2.3