aboutsummaryrefslogtreecommitdiff
path: root/src/model
Commit message (Collapse)AuthorAgeFilesLines
...
* Fix resync queue to not drop itemsAlex Auvolat2022-03-141-16/+21
|
* Spawn a single resync workerAlex Auvolat2022-03-141-11/+7
|
* Add appropriate fsync() calls in write_blockAlex Auvolat2022-03-141-2/+13
| | | | to ensure that data is persisted properly
* Implement exponential backoff for resync retriesfeature/opentelemetrybug/resync-exponential-backoffAlex Auvolat2022-03-142-5/+98
|
* Add wrapper over sled tree to count items (used for big queues)Alex Auvolat2022-03-142-2/+6
|
* Bump version to 0.7 because of incompatible NetappAlex Auvolat2022-03-141-4/+4
|
* Add metrics to API endpointAlex Auvolat2022-03-141-4/+7
|
* Refactor how durations are measuredAlex Auvolat2022-03-141-19/+14
|
* Update to Netapp 0.4 which supports distributed tracingAlex Auvolat2022-03-142-3/+24
|
* Add tracing integration with opentelemetryAlex Auvolat2022-03-143-5/+7
|
* Add many metrics in table/ and rpc/Alex Auvolat2022-03-144-4/+146
|
* Implement {Put,Get,Delete}BucketCors and CORS in generalAlex Auvolat2022-01-242-7/+26
| | | | | | - OPTIONS request against API endpoint - Returning corresponding CORS headers on API calls - Returning corresponding CORS headers on website GET's
* Add date verification to presigned urls (#196)trinity-1686a2022-01-181-5/+1
| | | | | | | | | | fix #96 fix #162 by returning Forbidden instead Bad Request Co-authored-by: Trinity Pointard <trinity.pointard@gmail.com> Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/196 Co-authored-by: trinity-1686a <trinity.pointard@gmail.com> Co-committed-by: trinity-1686a <trinity.pointard@gmail.com>
* Implement UploadPartCopyAlex Auvolat2022-01-131-0/+14
|
* Implement ListMultipartUploads (#171)Quentin2022-01-121-3/+11
| | | | | | | | | | | | | | | | | | | | | | | Implement ListMultipartUploads, also refactor ListObjects and ListObjectsV2. It took me some times as I wanted to propose the following things: - Using an iterator instead of the loop+goto pattern. I find it easier to read and it should enable some optimizations. For example, when consuming keys of a common prefix, we do many [redundant checks](https://git.deuxfleurs.fr/Deuxfleurs/garage/src/branch/main/src/api/s3_list.rs#L125-L156) while the only thing to do is to [check if the following key is still part of the common prefix](https://git.deuxfleurs.fr/Deuxfleurs/garage/src/branch/feature/s3-multipart-compat/src/api/s3_list.rs#L476). - Try to name things (see ExtractionResult and RangeBegin enums) and to separate concerns (see ListQuery and Accumulator) - An IO closure to make unit tests possibles. - Unit tests, to track regressions and document how to interact with the code - Integration tests with `s3api`. In the future, I would like to move them in Rust with the aws rust SDK. Merging of the logic of ListMultipartUploads and ListObjects was not a goal but a consequence of the previous modifications. Some points that we might want to discuss: - ListObjectsV1, when using pagination and delimiters, has a weird behavior (it lists multiple times the same prefix) with `aws s3api` due to the fact that it can not use our optimization to skip the whole prefix. It is independant from my refactor and can be tested with the commented `s3api` tests in `test-smoke.sh`. It probably has the same weird behavior on the official AWS S3 implementation. - Considering ListMultipartUploads, I had to "abuse" upload id marker to support prefix skipping. I send an `upload-id-marker` with the hardcoded value `include` to emulate your "including" token. - Some ways to test ListMultipartUploads with existing software (my tests are limited to s3api for now). Co-authored-by: Quentin Dufour <quentin@deuxfleurs.fr> Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/171 Co-authored-by: Quentin <quentin@dufour.io> Co-committed-by: Quentin <quentin@dufour.io>
* Implement CreateBucketAlex Auvolat2022-01-052-4/+13
|
* Small changes in key model and refactoringnew-bucketsAlex Auvolat2022-01-043-78/+81
|
* New buckets for 0.6.0: make bucket id a SK and not a HK, CLI updatesAlex Auvolat2022-01-042-10/+24
|
* New buckets for 0.6.0: small changesAlex Auvolat2022-01-044-35/+64
| | | | | | | | | | | | - Fix bucket delete - fix merge of bucket creation date - Replace deletable with option in aliases Rationale: if two aliases point to conflicting bucket, resolving by making an arbitrary choice risks making data accessible when it shouldn't be. We'd rather resolve to deleting the alias until someone puts it back.
* Rename and change query filtersAlex Auvolat2022-01-042-6/+6
|
* Refactor logic for setting/unsetting aliasesAlex Auvolat2022-01-044-60/+441
|
* Remove website redirectsAlex Auvolat2022-01-042-10/+4
|
* Some movement of helper code and refactoring of error handlingAlex Auvolat2022-01-047-19/+64
|
* New buckets for 0.6.0: fix model and migrationAlex Auvolat2022-01-047-63/+88
|
* New buckets for 0.6.0: small fixes, including:Alex Auvolat2022-01-046-23/+78
| | | | | | | | - ensure bucket names are correct aws s3 names - when making aliases, ensure timestamps of links in both ways are the same - fix small remarks by trinity - don't have a separate website_access field
* New buckets for 0.6.0: migration code and build filesAlex Auvolat2022-01-046-1/+228
|
* Model changesAlex Auvolat2022-01-044-7/+33
|
* Implement bucket alias and bucket unaliasAlex Auvolat2022-01-041-7/+21
|
* New model for bucketsAlex Auvolat2022-01-0410-108/+291
|
* Make table name a const in traitAlex Auvolat2021-12-156-17/+12
|
* Add compression using zstd (#173)trinity-1686a2021-12-152-36/+166
| | | | | | | | | fix #27 Co-authored-by: Trinity Pointard <trinity.pointard@gmail.com> Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/173 Co-authored-by: trinity-1686a <trinity.pointard@gmail.com> Co-committed-by: trinity-1686a <trinity.pointard@gmail.com>
* Improve how node roles are assigned in Garagev0.5-beta1Alex Auvolat2021-11-161-4/+5
| | | | | | | | | | | | | | | | | - change the terminology: the network configuration becomes the role table, the configuration of a nodes becomes a node's role - the modification of the role table takes place in two steps: first, changes are staged in a CRDT data structure. Then, once the user is happy with the changes, they can commit them all at once (or revert them). - update documentation - fix tests - implement smarter partition assignation algorithm This patch breaks the format of the network configuration: when migrating, the cluster will be in a state where no roles are assigned. All roles must be re-assigned and commited at once. This migration should not pose an issue.
* Implement GC delay for table dataAlex Auvolat2021-11-081-8/+2
|
* Safety: never voluntarily delete block in 10min interval after RC reaches zeroAlex Auvolat2021-11-081-48/+180
|
* Add tranquilizer mechanism to improve on token bucket mechanismtranquilityAlex Auvolat2021-11-041-25/+32
|
* Fix race in block resyncAlex Auvolat2021-10-271-9/+8
|
* Refactoring on repair commandscli-verify-integrityAlex Auvolat2021-10-271-40/+30
|
* add cli parameter to verify local bloc integrityTrinity Pointard2021-10-271-29/+92
| | | | | reuse code for listing local blocks add disk i/o speed limit on integrity check
* Use published netapp crate instead of git repoAlex Auvolat2021-10-261-1/+2
|
* Fix clippy lints (fix #121)Alex Auvolat2021-10-261-6/+6
|
* Refactor block manager code, and hopefully fix deadlockAlex Auvolat2021-10-251-170/+247
|
* Improve CLI, adapt tests, update documentationAlex Auvolat2021-10-252-7/+4
|
* Improvements to CLI and various fixes for netapp versionAlex Auvolat2021-10-222-17/+19
| | | | Discovery via consul, persist peer list to file
* First port of Garage to NetappAlex Auvolat2021-10-223-81/+82
|
* Prepare for v0.3.0 and add migration path from v0.2.1.xv0.3.0Alex Auvolat2021-05-281-4/+4
|
* Many improvements on ring/replication and its configuration:Alex Auvolat2021-05-281-8/+13
| | | | | | | | | | | | | | | | | | | | - Explicit "replication_mode" configuration parameters that takes either "none", "2" or "3" as values, instead of letting user configure replication factor themselves. These are presets whose corresponding replication/quorum values can be found in replication/mode.rs - Explicit support for single-node and two-node deployments (number of nodes must be at least "replication_mode", with "none" we can have only one node) - Ring is now stored much more compactly with 256*8 + n*32 bytes, instead of 256*32 bytes - Support for gateway-only nodes that do not store data (these nodes still need a metadata_directory to store the list of bucket and keys since those are stored on all nodes; it also technically needs a data_directory to start but it will stay empty unless we have bugs)
* rename types to CamelCaseTrinity Pointard2021-05-036-36/+36
|
* fix clippy warnings on modelTrinity Pointard2021-05-035-38/+38
|
* change a few commentsAlex Auvolat2021-04-275-7/+7
|
* change some more comments and revert changes on TableSchemaTrinity Pointard2021-04-273-11/+2
|
generated by cgit v1.2.3 (git 2.25.1) at 2024-11-18 03:49:34 +0000