aboutsummaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* Implement UploadPartCopyAlex Auvolat2022-01-137-56/+416
|
* Implement x-amz-copy-if-xxx copy preconditions and return more headers on ↵Alex Auvolat2022-01-132-0/+107
| | | | copy (fix #187)
* Implement ListMultipartUploads (#171)Quentin2022-01-1217-264/+1266
| | | | | | | | | | | | | | | | | | | | | | | Implement ListMultipartUploads, also refactor ListObjects and ListObjectsV2. It took me some times as I wanted to propose the following things: - Using an iterator instead of the loop+goto pattern. I find it easier to read and it should enable some optimizations. For example, when consuming keys of a common prefix, we do many [redundant checks](https://git.deuxfleurs.fr/Deuxfleurs/garage/src/branch/main/src/api/s3_list.rs#L125-L156) while the only thing to do is to [check if the following key is still part of the common prefix](https://git.deuxfleurs.fr/Deuxfleurs/garage/src/branch/feature/s3-multipart-compat/src/api/s3_list.rs#L476). - Try to name things (see ExtractionResult and RangeBegin enums) and to separate concerns (see ListQuery and Accumulator) - An IO closure to make unit tests possibles. - Unit tests, to track regressions and document how to interact with the code - Integration tests with `s3api`. In the future, I would like to move them in Rust with the aws rust SDK. Merging of the logic of ListMultipartUploads and ListObjects was not a goal but a consequence of the previous modifications. Some points that we might want to discuss: - ListObjectsV1, when using pagination and delimiters, has a weird behavior (it lists multiple times the same prefix) with `aws s3api` due to the fact that it can not use our optimization to skip the whole prefix. It is independant from my refactor and can be tested with the commented `s3api` tests in `test-smoke.sh`. It probably has the same weird behavior on the official AWS S3 implementation. - Considering ListMultipartUploads, I had to "abuse" upload id marker to support prefix skipping. I send an `upload-id-marker` with the hardcoded value `include` to emulate your "including" token. - Some ways to test ListMultipartUploads with existing software (my tests are limited to s3api for now). Co-authored-by: Quentin Dufour <quentin@deuxfleurs.fr> Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/171 Co-authored-by: Quentin <quentin@dufour.io> Co-committed-by: Quentin <quentin@dufour.io>
* Add documentation to migrate to 0.6update-documentationcreate-delete-bucketsAlex Auvolat2022-01-102-0/+47
|
* Clarify error messageAlex Auvolat2022-01-102-9/+10
|
* Fix trivial bug in CLIAlex Auvolat2022-01-101-2/+6
|
* Update documentationAlex Auvolat2022-01-072-9/+4
|
* Fix unit testsAlex Auvolat2022-01-052-12/+44
|
* Fix some error codesAlex Auvolat2022-01-058-36/+53
|
* Small CLI changesAlex Auvolat2022-01-051-5/+8
|
* Implement DeleteBucketAlex Auvolat2022-01-053-6/+116
|
* Implement CreateBucketAlex Auvolat2022-01-055-28/+166
|
* Implement key allow|deny --create-bucketAlex Auvolat2022-01-052-0/+48
|
* Small changes in key model and refactoringnew-bucketsAlex Auvolat2022-01-047-103/+119
|
* New buckets for 0.6.0: make bucket id a SK and not a HK, CLI updatesAlex Auvolat2022-01-049-47/+86
|
* New buckets for 0.6.0: small changesAlex Auvolat2022-01-0412-62/+86
| | | | | | | | | | | | - Fix bucket delete - fix merge of bucket creation date - Replace deletable with option in aliases Rationale: if two aliases point to conflicting bucket, resolving by making an arbitrary choice risks making data accessible when it shouldn't be. We'd rather resolve to deleting the alias until someone puts it back.
* Rename and change query filtersAlex Auvolat2022-01-043-8/+8
|
* Hopefully fix Nix buildAlex Auvolat2022-01-042-2/+3
|
* Refactor logic for setting/unsetting aliasesAlex Auvolat2022-01-045-413/+570
|
* Remove website redirectsAlex Auvolat2022-01-044-20/+14
|
* Some movement of helper code and refactoring of error handlingAlex Auvolat2022-01-0417-66/+137
|
* New buckets for 0.6.0: fix model and migrationAlex Auvolat2022-01-0415-96/+174
|
* Add must_use to some CRDT functionsAlex Auvolat2022-01-043-1/+3
|
* Fix forgotten flagAlex Auvolat2022-01-041-1/+1
|
* New buckets for 0.6.0: documentation and build filesAlex Auvolat2022-01-042-0/+12
|
* New buckets for 0.6.0: small fixes, including:Alex Auvolat2022-01-0418-107/+243
| | | | | | | | - ensure bucket names are correct aws s3 names - when making aliases, ensure timestamps of links in both ways are the same - fix small remarks by trinity - don't have a separate website_access field
* More complete output to bucket info and key infoAlex Auvolat2022-01-043-35/+147
|
* New buckets for 0.6.0: migration code and build filesAlex Auvolat2022-01-0411-57/+435
|
* Model changesAlex Auvolat2022-01-0414-57/+119
|
* Implement bucket alias and bucket unaliasAlex Auvolat2022-01-045-10/+252
|
* New model for bucketsAlex Auvolat2022-01-0438-485/+1162
|
* Make table name a const in traitAlex Auvolat2021-12-1513-70/+70
|
* update s3 compatibility list (#177)trinity-1686a2021-12-152-16/+22
| | | | | | | Co-authored-by: Trinity Pointard <trinity.pointard@gmail.com> Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/177 Co-authored-by: trinity-1686a <trinity.pointard@gmail.com> Co-committed-by: trinity-1686a <trinity.pointard@gmail.com>
* less strict ListBuckets (#178)trinity-1686a2021-12-151-5/+1
| | | | | | | | | fix #175 Co-authored-by: Trinity Pointard <trinity.pointard@gmail.com> Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/178 Co-authored-by: trinity-1686a <trinity.pointard@gmail.com> Co-committed-by: trinity-1686a <trinity.pointard@gmail.com>
* Add compression using zstd (#173)trinity-1686a2021-12-158-53/+377
| | | | | | | | | fix #27 Co-authored-by: Trinity Pointard <trinity.pointard@gmail.com> Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/173 Co-authored-by: trinity-1686a <trinity.pointard@gmail.com> Co-committed-by: trinity-1686a <trinity.pointard@gmail.com>
* BucketWebsite (#174)trinity-1686a2021-12-154-4/+318
| | | | | | | | | | | fix #77 this does not store anything but a on/off switch for website, and does not implement GetBucketWebsite as it would require storing more. GetBucketWebsite should be pretty easy to implement once data is stored though. Co-authored-by: Trinity Pointard <trinity.pointard@gmail.com> Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/174 Co-authored-by: trinity-1686a <trinity.pointard@gmail.com> Co-committed-by: trinity-1686a <trinity.pointard@gmail.com>
* Add a second plotdoc/bench-multi-dcQuentin Dufour2021-12-132-1/+21
|
* Update benchmarkdoc/fixQuentin Dufour2021-12-091-0/+0
|
* Typos and dead linksQuentin Dufour2021-12-084-4/+4
|
* Add a benchmark page with a first benchmarkQuentin Dufour2021-12-083-0/+62
|
* S3 compatibility targetv0.5.0.1Alex Auvolat2021-12-062-0/+106
|
* Reorganize and improve documentationAlex Auvolat2021-12-0611-101/+127
|
* WIP front page garageQuentin Dufour2021-12-067-51/+116
|
* add proper request router for s3 api (#163)trinity-1686a2021-12-067-185/+1476
| | | | | | | | | | | | | fix #161 Current request router was organically grown, and is getting messier and messier with each addition. This router cover exaustively existing API endpoints (with exceptions listed in [#161(comment)](https://git.deuxfleurs.fr/Deuxfleurs/garage/issues/161#issuecomment-1773) either because new and old api endpoint can't feasabily be differentied, or it's more lambda than s3). Co-authored-by: Trinity Pointard <trinity.pointard@gmail.com> Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/163 Reviewed-by: Alex <alex@adnab.me> Co-authored-by: trinity-1686a <trinity.pointard@gmail.com> Co-committed-by: trinity-1686a <trinity.pointard@gmail.com>
* Remove TODO and genkeys.shAlex Auvolat2021-12-062-110/+0
|
* Improved handling of HTTP rangestrinity-1686a2021-11-295-14/+46
| | | | | | | | | | | - correct HTTP code when range syntax is invalid (fix #140) - when multiple ranges are given, simply ignore and send whole file Co-authored-by: Trinity Pointard <trinity.pointard@gmail.com> Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/157 Reviewed-by: Alex <alex@adnab.me> Co-authored-by: trinity-1686a <trinity.pointard@gmail.com> Co-committed-by: trinity-1686a <trinity.pointard@gmail.com>
* In ListBuckets, hide entry if no permsfeature/hide-buckets-without-permQuentin Dufour2021-11-222-0/+16
|
* Start socat only oncev0.5.0bug/script-socatQuentin Dufour2021-11-171-4/+4
| | | | Fixes #124
* Improve how node roles are assigned in Garagev0.5-beta1Alex Auvolat2021-11-1642-557/+1430
| | | | | | | | | | | | | | | | | - change the terminology: the network configuration becomes the role table, the configuration of a nodes becomes a node's role - the modification of the role table takes place in two steps: first, changes are staged in a CRDT data structure. Then, once the user is happy with the changes, they can commit them all at once (or revert them). - update documentation - fix tests - implement smarter partition assignation algorithm This patch breaks the format of the network configuration: when migrating, the cluster will be in a state where no roles are assigned. All roles must be re-assigned and commited at once. This migration should not pose an issue.
* update doc and commentsTrinity Pointard2021-11-163-9/+12
|
generated by cgit v1.2.3 (git 2.25.1) at 2024-11-01 23:28:49 +0000