diff options
| -rw-r--r-- | content/blog/2022-roadmap.md | 62 |
1 files changed, 62 insertions, 0 deletions
diff --git a/content/blog/2022-roadmap.md b/content/blog/2022-roadmap.md new file mode 100644 index 0000000..52f5d72 --- /dev/null +++ b/content/blog/2022-roadmap.md @@ -0,0 +1,62 @@ ++++ +title="Our roadmap towards v1.0" +date=2022-04-10 ++++ + +*a* + +<!-- more --> + +--- + + + +While we hope that Garage in its current state inspired you, we also understand that you may be curious about what will come next! +As per our road towards v1.0, we identified the remaining work and categorized it under one of these 3 domains: *Feature completeness*, *Understandability and manageability*, and *Correctness*. + +## Feature completeness + +Most importantly, we still need to fix some corner cases on advanced S3 endpoints (eg. [#263](https://git.deuxfleurs.fr/Deuxfleurs/garage/issues), [#248](https://git.deuxfleurs.fr/Deuxfleurs/garage/issues/248), [#204](https://git.deuxfleurs.fr/Deuxfleurs/garage/issues/204)). Based on community feedbacks, we might also consider implementing additional endpoints (eg. [#166](https://git.deuxfleurs.fr/Deuxfleurs/garage/issues/166)) or quotas (eg. [#71](https://git.deuxfleurs.fr/Deuxfleurs/garage/issues/71)) but we can't make any promise (sorry!). + +We also made a series of observation (the S3 API is not adapted to handle multiple small objects, many software require a database, we already have an internal database for metadata) which makes us believe that Garage could leverage its internal database system to provide a simple key-value store. We have already written [an API draft](https://p.adnab.me/code/#/2/code/view/eUNPbfoUrMbCY+CoMXaqed4jmWlmvWALHNDcfuM-O5o/embed/present/) for an API we named K2V. In the following months, we will then try to implement it and merge it if it makes sense. Spiritually, we would like it to be close to the original [Amazon Dynamo paper](https://www.allthingsdistributed.com/files/amazon-dynamo-sosp2007.pdf), or if you prefer approximative comparisons, K2V could be to Cassandra what sqlite is to PostgreSQL. +If you worry about feature bloat, be ensured that we do not plan to extend Garage beyond these points! + +## Understandability and manageability + +Based on community feedback (eg. [#228](https://git.deuxfleurs.fr/Deuxfleurs/garage/issues/228), [#221](https://git.deuxfleurs.fr/Deuxfleurs/garage/issues/221), [#217](https://git.deuxfleurs.fr/Deuxfleurs/garage/issues/217)), we have identified some points on which we want to dedicate work: *Consistency Model*, *Administration*, *Storage Model*, and *Ecosystem*. + +Garage has currently 2 API: S3 and Admin, with different *Consistency Models*. S3 API's consistency is aligned on [Amazon's new S3 Strong Consistency](https://aws.amazon.com/s3/consistency/) while Admin API's eventual consistency is not yet specified. We want to document first the Admin API's consistency, then we could make S3 consistency explanation more approachable. + +The Admin API is also currently only exposed through our custom RPC endpoint. We would like to expose it through a REST endpoint ([#231](https://git.deuxfleurs.fr/Deuxfleurs/garage/issues/231)) to ease *Administration*. This REST API would make possible to build a web interface to manage Garage ([#232](https://git.deuxfleurs.fr/Deuxfleurs/garage/issues/232)). + +When people discover Garage, questions also arise around its *Storage Model*, eg. what reliability and density shoud they expect depending on their considered configuration. Ideally, we would like to write a calculator similar to [Minio's one](https://min.io/product/erasure-code-calculator) to help Garage operators understand the properties of their cluster. + +Finally, Garage exists in an *Ecosystem*, consisting of multiple alternative software (eg. Ceph, Minio, SeaweedFS, IPFS Cluster), with different needs (research, customer services, archiving), made of various hardware (from a Raspberry Pi to a 64+ disk blade server). We want to situate Garage in its ecosystem while identifying, promoting and documenting some "Garage success story". + +<!--2) Explaining how Garage can take its place in the existing ecosystem, including among the other distributed storage systems, but also in term of uses cases and deployments (how does it perform at scale, with which hardware, for which application, etc.) 3) Make possible to manage Garage from a REST API, possibly write a web GUI to make administration easier, 4) help people understand the reliability and storage density they will have for a specific Garage deployment, if possible through a simulator, 5) we might consider adding a system of quota to protect a cluster from a misbehaving user. 6) Integration in ecosystems --> + +## Correctness + +We know in theory that Garage's design works and horizontally scales. +But we still need to make sure that in practise our implementation is correct, and thus features these defined properties. + +[Jepsen](https://jepsen.io/) is a well-known tool to test distributed system properties by simulating some system states and sequencing packets in a specific order. +We would like to learn it and apply it to Garage to better convince ourselves that we + + +Consider control theory, speak about tranquilizer ([#145](https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/145)). + + +Horizontal scaling made possible no leader also no erasure coding. +Track bottlenecks, instability and overload. +We also plan to deploy Garage on multiple clusters and do a large serie of benchmarks. + +<!-- non goals: erasure coding, byzantine tolerance ; do we want quota? --> + +## Conclusion + +Please note that this roadmap is purely indicative, we are not committing to deliver these features. +Contributions are however welcome, so get in touch (on our Matrix channel) if you want to implement one of them. + +We also don't know when v1.0 will be released, except "when it will be ready", but we would be happy if it could be by the end of 2022. +See you soon for the next release! |