aboutsummaryrefslogtreecommitdiff
path: root/src/api/s3_bucket.rs
Commit message (Collapse)AuthorAgeFilesLines
* Implement ListMultipartUploads (#171)Quentin2022-01-121-1/+2
| | | | | | | | | | | | | | | | | | | | | | | Implement ListMultipartUploads, also refactor ListObjects and ListObjectsV2. It took me some times as I wanted to propose the following things: - Using an iterator instead of the loop+goto pattern. I find it easier to read and it should enable some optimizations. For example, when consuming keys of a common prefix, we do many [redundant checks](https://git.deuxfleurs.fr/Deuxfleurs/garage/src/branch/main/src/api/s3_list.rs#L125-L156) while the only thing to do is to [check if the following key is still part of the common prefix](https://git.deuxfleurs.fr/Deuxfleurs/garage/src/branch/feature/s3-multipart-compat/src/api/s3_list.rs#L476). - Try to name things (see ExtractionResult and RangeBegin enums) and to separate concerns (see ListQuery and Accumulator) - An IO closure to make unit tests possibles. - Unit tests, to track regressions and document how to interact with the code - Integration tests with `s3api`. In the future, I would like to move them in Rust with the aws rust SDK. Merging of the logic of ListMultipartUploads and ListObjects was not a goal but a consequence of the previous modifications. Some points that we might want to discuss: - ListObjectsV1, when using pagination and delimiters, has a weird behavior (it lists multiple times the same prefix) with `aws s3api` due to the fact that it can not use our optimization to skip the whole prefix. It is independant from my refactor and can be tested with the commented `s3api` tests in `test-smoke.sh`. It probably has the same weird behavior on the official AWS S3 implementation. - Considering ListMultipartUploads, I had to "abuse" upload id marker to support prefix skipping. I send an `upload-id-marker` with the hardcoded value `include` to emulate your "including" token. - Some ways to test ListMultipartUploads with existing software (my tests are limited to s3api for now). Co-authored-by: Quentin Dufour <quentin@deuxfleurs.fr> Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/171 Co-authored-by: Quentin <quentin@dufour.io> Co-committed-by: Quentin <quentin@dufour.io>
* Clarify error messageAlex Auvolat2022-01-101-1/+2
|
* Fix unit testsAlex Auvolat2022-01-051-6/+16
|
* Fix some error codesAlex Auvolat2022-01-051-8/+2
|
* Implement DeleteBucketAlex Auvolat2022-01-051-2/+96
|
* Implement CreateBucketAlex Auvolat2022-01-051-2/+144
|
* Small changes in key model and refactoringnew-bucketsAlex Auvolat2022-01-041-4/+4
|
* New buckets for 0.6.0: make bucket id a SK and not a HK, CLI updatesAlex Auvolat2022-01-041-12/+8
|
* New buckets for 0.6.0: small changesAlex Auvolat2022-01-041-4/+4
| | | | | | | | | | | | - Fix bucket delete - fix merge of bucket creation date - Replace deletable with option in aliases Rationale: if two aliases point to conflicting bucket, resolving by making an arbitrary choice risks making data accessible when it shouldn't be. We'd rather resolve to deleting the alias until someone puts it back.
* Fix forgotten flagAlex Auvolat2022-01-041-1/+1
|
* New buckets for 0.6.0: small fixes, including:Alex Auvolat2022-01-041-1/+1
| | | | | | | | - ensure bucket names are correct aws s3 names - when making aliases, ensure timestamps of links in both ways are the same - fix small remarks by trinity - don't have a separate website_access field
* New model for bucketsAlex Auvolat2022-01-041-7/+55
|
* In ListBuckets, hide entry if no permsfeature/hide-buckets-without-permQuentin Dufour2021-11-221-0/+1
|
* Add basic support for the "Versioning" commandQuentin Dufour2021-05-141-0/+13
|
* Improved XML serializationbetter_xmlAlex Auvolat2021-05-061-77/+15
| | | | | | - Use quick_xml and serde for all XML response returned by the S3 API. - Include tests for all structs used to generate XML - Remove old manual XML escaping function which was unsafe
* rename types to CamelCaseTrinity Pointard2021-05-031-3/+3
|
* S3 API: support ListBucketsQuentin Dufour2021-05-031-0/+87
|
* Many S3 compatibility improvements:v0.2.1.5Alex Auvolat2021-04-281-0/+24
- return XML errors - implement AuthorizationHeaderMalformed error to redirect clients to correct location (used by minio client) - implement GetBucketLocation - fix DeleteObjects XML parsing and response