aboutsummaryrefslogtreecommitdiff
path: root/src/garage/cli
Commit message (Collapse)AuthorAgeFilesLines
...
* Merge branch 'lx-perf-improvements' into netapp-stream-bodyAlex Auvolat2022-09-081-56/+56
|\
| * Move version back into utilimprove-depsAlex Auvolat2022-09-071-1/+1
| |
| * Fix mergeAlex Auvolat2022-09-071-4/+4
| |
| * Merge branch 'main' into improve-depsAlex Auvolat2022-09-071-0/+19
| |\
| * | Move GIT_VERSION injection later in build chain to reduce build timesAlex Auvolat2022-09-071-52/+52
| | |
* | | Merge branch 'lx-perf-improvements' into netapp-stream-bodyAlex Auvolat2022-09-061-0/+19
|\ \ \ | | |/ | |/|
| * | Ability to have up to 4 concurrently working resync workersAlex Auvolat2022-09-021-1/+4
| | |
| * | Ability to dynamically set resync tranquilityAlex Auvolat2022-09-021-0/+16
| |/
* | Merge branch 'lx-perf-improvements' into netapp-stream-bodyAlex Auvolat2022-08-291-50/+51
|\|
| * Configure structopt to report the right versionv0.7.2_ci-test-2bug/reported-versionQuentin Dufour2022-08-111-50/+51
| | | | | | | | | | | | | | | | | | | | By default, structopt reports the value provided by the env var CARGO_PKG_VERSION, feeded by Cargo when reading Cargo.toml. However for Garage we use a versioning based on git, so we often report a version that is behind the real version. In this commit, we create garage_util::version::garage() that reports the right version and configure all structopt subcommands to call this function instead of using the env var.
* | First adaptation to WIP netapp with streaming bodyAlex Auvolat2022-07-292-6/+6
|/
* Background task manager (#332)Alex2022-07-083-3/+118
| | | | | | | | | | | | | | | | | | | | | - [x] New background worker trait - [x] Adapt all current workers to use new API - [x] Command to list currently running workers, and whether they are active, idle, or dead - [x] Error reporting - Optimizations - [x] Merkle updater: several items per iteration - [ ] Use `tokio::task::spawn_blocking` where appropriate so that CPU-intensive tasks don't block other things going on - scrub: - [x] have only one worker with a channel to start/pause/cancel - [x] automatic scrub - [x] ability to view and change tranquility from CLI - [x] persistence of a few info - [ ] Testing Co-authored-by: Alex Auvolat <alex@adnab.me> Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/332 Co-authored-by: Alex <alex@adnab.me> Co-committed-by: Alex <alex@adnab.me>
* improve internal item counter mechanisms and implement bucket quotas (#326)Alex2022-06-153-6/+96
| | | | | | | | | | | | | | | | - [x] Refactoring of internal counting API - [x] Repair procedure for counters (it's an offline procedure!!!) - [x] New counter for objects in buckets - [x] Add quotas to buckets struct - [x] Add CLI to manage bucket quotas - [x] Add admin API to manage bucket quotas - [x] Apply quotas by adding checks on put operations - [x] Proof-read Co-authored-by: Alex Auvolat <alex@adnab.me> Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/326 Co-authored-by: Alex <alex@adnab.me> Co-committed-by: Alex <alex@adnab.me>
* First version of admin API (#298)Alex2022-05-241-43/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | **Spec:** - [x] Start writing - [x] Specify all layout endpoints - [x] Specify all endpoints for operations on keys - [x] Specify all endpoints for operations on key/bucket permissions - [x] Specify all endpoints for operations on buckets - [x] Specify all endpoints for operations on bucket aliases View rendered spec at <https://git.deuxfleurs.fr/Deuxfleurs/garage/src/branch/admin-api/doc/drafts/admin-api.md> **Code:** - [x] Refactor code for admin api to use common api code that was created for K2V **General endpoints:** - [x] Metrics - [x] GetClusterStatus - [x] ConnectClusterNodes - [x] GetClusterLayout - [x] UpdateClusterLayout - [x] ApplyClusterLayout - [x] RevertClusterLayout **Key-related endpoints:** - [x] ListKeys - [x] CreateKey - [x] ImportKey - [x] GetKeyInfo - [x] UpdateKey - [x] DeleteKey **Bucket-related endpoints:** - [x] ListBuckets - [x] CreateBucket - [x] GetBucketInfo - [x] DeleteBucket - [x] PutBucketWebsite - [x] DeleteBucketWebsite **Operations on key/bucket permissions:** - [x] BucketAllowKey - [x] BucketDenyKey **Operations on bucket aliases:** - [x] GlobalAliasBucket - [x] GlobalUnaliasBucket - [x] LocalAliasBucket - [x] LocalUnaliasBucket **And also:** - [x] Separate error type for the admin API (this PR includes a quite big refactoring of error handling) - [x] Add management of website access - [ ] Check that nothing is missing wrt what can be done using the CLI - [ ] Improve formatting of the spec - [x] Make sure everyone is cool with the API design Fix #231 Fix #295 Co-authored-by: Alex Auvolat <alex@adnab.me> Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/298 Co-authored-by: Alex <alex@adnab.me> Co-committed-by: Alex <alex@adnab.me>
* Add a K2V client library and CLI (#303)trinity-1686a2022-05-183-29/+3
| | | | | | | | | lib.rs could use getting split in modules, but I'm not sure how exactly Co-authored-by: trinity-1686a <trinity@deuxfleurs.fr> Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/303 Co-authored-by: trinity-1686a <trinity.pointard@gmail.com> Co-committed-by: trinity-1686a <trinity.pointard@gmail.com>
* First implementation of K2V (#293)Alex2022-05-101-3/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | **Specification:** View spec at [this URL](https://git.deuxfleurs.fr/Deuxfleurs/garage/src/branch/k2v/doc/drafts/k2v-spec.md) - [x] Specify the structure of K2V triples - [x] Specify the DVVS format used for causality detection - [x] Specify the K2V index (just a counter of number of values per partition key) - [x] Specify single-item endpoints: ReadItem, InsertItem, DeleteItem - [x] Specify index endpoint: ReadIndex - [x] Specify multi-item endpoints: InsertBatch, ReadBatch, DeleteBatch - [x] Move to JSON objects instead of tuples - [x] Specify endpoints for polling for updates on single values (PollItem) **Implementation:** - [x] Table for K2V items, causal contexts - [x] Indexing mechanism and table for K2V index - [x] Make API handlers a bit more generic - [x] K2V API endpoint - [x] K2V API router - [x] ReadItem - [x] InsertItem - [x] DeleteItem - [x] PollItem - [x] ReadIndex - [x] InsertBatch - [x] ReadBatch - [x] DeleteBatch **Testing:** - [x] Just a simple Python script that does some requests to check visually that things are going right (does not contain parsing of results or assertions on returned values) - [x] Actual tests: - [x] Adapt testing framework - [x] Simple test with InsertItem + ReadItem - [x] Test with several Insert/Read/DeleteItem + ReadIndex - [x] Test all combinations of return formats for ReadItem - [x] Test with ReadBatch, InsertBatch, DeleteBatch - [x] Test with PollItem - [x] Test error codes - [ ] Fix most broken stuff - [x] test PollItem broken randomly - [x] when invalid causality tokens are given, errors should be 4xx not 5xx **Improvements:** - [x] Descending range queries - [x] Specify - [x] Implement - [x] Add test - [x] Batch updates to index counter - [x] Put K2V behind `k2v` feature flag Co-authored-by: Alex Auvolat <alex@adnab.me> Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/293 Co-authored-by: Alex <alex@adnab.me> Co-committed-by: Alex <alex@adnab.me>
* Fix `layout show` to not show changes when there are no changes (#297)Alex2022-05-092-5/+22
| | | | | | | | | fixes #295, partially Co-authored-by: Alex Auvolat <alex@adnab.me> Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/297 Co-authored-by: Alex <alex@adnab.me> Co-committed-by: Alex <alex@adnab.me>
* Allow `garage layout assign` to assign to several nodes at onceAlex Auvolat2022-03-242-46/+56
|
* Small change to partition assignation algorithmAlex Auvolat2022-03-241-9/+9
| | | | | | This change helps ensure that nodes for each partition are spread over all datacenters, a property that wasn't ensured previously when going from a 2 DC deployment to a 3 DC deployment
* Add tracing integration with opentelemetryAlex Auvolat2022-03-141-2/+0
|
* Hide deleted key in bucket info (fix #211)Alex Auvolat2022-02-021-0/+3
|
* Allow setting index document and error document on the CLIAlex Auvolat2022-01-131-0/+8
|
* Small CLI changesAlex Auvolat2022-01-051-5/+8
|
* Implement key allow|deny --create-bucketAlex Auvolat2022-01-051-0/+18
|
* Small changes in key model and refactoringnew-bucketsAlex Auvolat2022-01-041-14/+17
|
* New buckets for 0.6.0: make bucket id a SK and not a HK, CLI updatesAlex Auvolat2022-01-042-15/+44
|
* New buckets for 0.6.0: small changesAlex Auvolat2022-01-042-4/+4
| | | | | | | | | | | | - Fix bucket delete - fix merge of bucket creation date - Replace deletable with option in aliases Rationale: if two aliases point to conflicting bucket, resolving by making an arbitrary choice risks making data accessible when it shouldn't be. We'd rather resolve to deleting the alias until someone puts it back.
* Some movement of helper code and refactoring of error handlingAlex Auvolat2022-01-041-6/+8
|
* New buckets for 0.6.0: small fixes, including:Alex Auvolat2022-01-042-2/+2
| | | | | | | | - ensure bucket names are correct aws s3 names - when making aliases, ensure timestamps of links in both ways are the same - fix small remarks by trinity - don't have a separate website_access field
* More complete output to bucket info and key infoAlex Auvolat2022-01-042-12/+69
|
* New buckets for 0.6.0: migration code and build filesAlex Auvolat2022-01-042-0/+25
|
* Model changesAlex Auvolat2022-01-043-4/+11
|
* Implement bucket alias and bucket unaliasAlex Auvolat2022-01-043-3/+44
|
* New model for bucketsAlex Auvolat2022-01-042-16/+44
|
* Improve how node roles are assigned in Garagev0.5-beta1Alex Auvolat2021-11-164-182/+452
| | | | | | | | | | | | | | | | | - change the terminology: the network configuration becomes the role table, the configuration of a nodes becomes a node's role - the modification of the role table takes place in two steps: first, changes are staged in a CRDT data structure. Then, once the user is happy with the changes, they can commit them all at once (or revert them). - update documentation - fix tests - implement smarter partition assignation algorithm This patch breaks the format of the network configuration: when migrating, the cluster will be in a state where no roles are assigned. All roles must be re-assigned and commited at once. This migration should not pose an issue.
* Safety: never voluntarily delete block in 10min interval after RC reaches zeroAlex Auvolat2021-11-081-2/+1
|
* Add tranquilizer mechanism to improve on token bucket mechanismtranquilityAlex Auvolat2021-11-041-3/+3
|
* Refactoring on repair commandscli-verify-integrityAlex Auvolat2021-10-271-3/+3
|
* add cli parameter to verify local bloc integrityTrinity Pointard2021-10-271-0/+7
| | | | | reuse code for listing local blocks add disk i/o speed limit on integrity check
* CLI: default rpc_hostv0.4-rc1Alex Auvolat2021-10-261-3/+5
|
* Fix clippy lints (fix #121)Alex Auvolat2021-10-261-1/+1
|
* Improve CLI, adapt tests, update documentationAlex Auvolat2021-10-255-0/+740
generated by cgit v1.2.3 (git 2.25.1) at 2024-11-25 17:35:22 +0000