diff options
author | Quentin Dufour <quentin@deuxfleurs.fr> | 2022-01-24 12:04:58 +0100 |
---|---|---|
committer | Quentin Dufour <quentin@deuxfleurs.fr> | 2022-01-24 12:04:58 +0100 |
commit | ba7be3f895ec187fcd798018f0aa323232625ff6 (patch) | |
tree | 611b9dfd334bed4d40be62af5d08b3b08169a031 /doc/book/src/connect | |
parent | 9374389f873ca0dc2306d81506fa7aef799e7c5a (diff) | |
download | garage-ba7be3f895ec187fcd798018f0aa323232625ff6.tar.gz garage-ba7be3f895ec187fcd798018f0aa323232625ff6.zip |
Add doc for Publii + Peertube
Diffstat (limited to 'doc/book/src/connect')
-rw-r--r-- | doc/book/src/connect/apps.md | 169 | ||||
-rw-r--r-- | doc/book/src/connect/publii.png | bin | 0 -> 137077 bytes | |||
-rw-r--r-- | doc/book/src/connect/websites.md | 39 |
3 files changed, 59 insertions, 149 deletions
diff --git a/doc/book/src/connect/apps.md b/doc/book/src/connect/apps.md index 14d69ef8..c9bba409 100644 --- a/doc/book/src/connect/apps.md +++ b/doc/book/src/connect/apps.md @@ -1,6 +1,19 @@ # Apps (Nextcloud, Peertube...) -In this section, we cover the following software: [Nextcloud](#nextcloud), [Peertube](#peertube), [Mastodon](#mastodon), [Matrix](#matrix) +In this section, we cover the following web applications: + +| Name | Status | Note | +|------|--------|------| +| [Nextcloud](#nextcloud) | ✅ | Both Primary Storage and External Storage are supported | +| [Peertube](#peertube) | ✅ | `base_url` must be set to the website endpoint | +| [Mastodon](#mastodon) | ❓ | | +| [Matrix](#matrix) | ✅ | Tested with `synapse-s3-storage-provider` | +| [Pixelfed](#pixelfed) | ❓ | | +| [Pleroma](#pleroma) | ❓ | | +| [Lemmy](#lemmy) | ❓ | | +| [Funkwhale](#funkwhale) | ❓ | | +| [Misskey](#misskey) | ❓ | | +| [Prismo](#prismo) | ❓ | | ## Nextcloud @@ -108,109 +121,8 @@ Do not change the `use_path_style` and `legacy_auth` entries, other configuratio Peertube proposes a clever integration of S3 by directly exposing its endpoint instead of proxifying requests through the application. In other words, Peertube is only responsible of the "control plane" and offload the "data plane" to Garage. -In return, this system is a bit harder to configure, especially with Garage that supports less feature than other older S3 backends. -We show that it is still possible to configure Garage with Peertube, allowing you to spread the load and the bandwidth usage on the Garage cluster. - -### Enable path-style access by patching Peertube - -First, you will need to apply a small patch on Peertube ([#4510](https://github.com/Chocobozzz/PeerTube/pull/4510)): - -```diff -From e3b4c641bdf67e07d406a1d49d6aa6b1fbce2ab4 Mon Sep 17 00:00:00 2001 -From: Martin Honermeyer <maze@strahlungsfrei.de> -Date: Sun, 31 Oct 2021 12:34:04 +0100 -Subject: [PATCH] Allow setting path-style access for object storage - ---- - config/default.yaml | 4 ++++ - config/production.yaml.example | 4 ++++ - server/initializers/config.ts | 1 + - server/lib/object-storage/shared/client.ts | 3 ++- - .../production/config/custom-environment-variables.yaml | 2 ++ - 5 files changed, 13 insertions(+), 1 deletion(-) - -diff --git a/config/default.yaml b/config/default.yaml -index cf9d69a6211..4efd56fb804 100644 ---- a/config/default.yaml -+++ b/config/default.yaml -@@ -123,6 +123,10 @@ object_storage: - # You can also use AWS_SECRET_ACCESS_KEY env variable - secret_access_key: '' - -+ # Reference buckets via path rather than subdomain -+ # (i.e. "my-endpoint.com/bucket" instead of "bucket.my-endpoint.com") -+ force_path_style: false -+ - # Maximum amount to upload in one request to object storage - max_upload_part: 2GB - -diff --git a/config/production.yaml.example b/config/production.yaml.example -index 70993bf57a3..9ca2de5f4c9 100644 ---- a/config/production.yaml.example -+++ b/config/production.yaml.example -@@ -121,6 +121,10 @@ object_storage: - # You can also use AWS_SECRET_ACCESS_KEY env variable - secret_access_key: '' - -+ # Reference buckets via path rather than subdomain -+ # (i.e. "my-endpoint.com/bucket" instead of "bucket.my-endpoint.com") -+ force_path_style: false -+ - # Maximum amount to upload in one request to object storage - max_upload_part: 2GB - -diff --git a/server/initializers/config.ts b/server/initializers/config.ts -index 8375bf4304c..d726c59a4b6 100644 ---- a/server/initializers/config.ts -+++ b/server/initializers/config.ts -@@ -91,6 +91,7 @@ const CONFIG = { - ACCESS_KEY_ID: config.get<string>('object_storage.credentials.access_key_id'), - SECRET_ACCESS_KEY: config.get<string>('object_storage.credentials.secret_access_key') - }, -+ FORCE_PATH_STYLE: config.get<boolean>('object_storage.force_path_style'), - VIDEOS: { - BUCKET_NAME: config.get<string>('object_storage.videos.bucket_name'), - PREFIX: config.get<string>('object_storage.videos.prefix'), -diff --git a/server/lib/object-storage/shared/client.ts b/server/lib/object-storage/shared/client.ts -index c9a61459336..eadad02f93f 100644 ---- a/server/lib/object-storage/shared/client.ts -+++ b/server/lib/object-storage/shared/client.ts -@@ -26,7 +26,8 @@ function getClient () { - accessKeyId: OBJECT_STORAGE.CREDENTIALS.ACCESS_KEY_ID, - secretAccessKey: OBJECT_STORAGE.CREDENTIALS.SECRET_ACCESS_KEY - } -- : undefined -+ : undefined, -+ forcePathStyle: CONFIG.OBJECT_STORAGE.FORCE_PATH_STYLE - }) - - logger.info('Initialized S3 client %s with region %s.', getEndpoint(), OBJECT_STORAGE.REGION, lTags()) -diff --git a/support/docker/production/config/custom-environment-variables.yaml b/support/docker/production/config/custom-environment-variables.yaml -index c7cd28e6521..a960bab0bc9 100644 ---- a/support/docker/production/config/custom-environment-variables.yaml -+++ b/support/docker/production/config/custom-environment-variables.yaml -@@ -54,6 +54,8 @@ object_storage: - - region: "PEERTUBE_OBJECT_STORAGE_REGION" - -+ force_path_style: "PEERTUBE_OBJECT_STORAGE_FORCE_PATH_STYLE" -+ - max_upload_part: - __name: "PEERTUBE_OBJECT_STORAGE_MAX_UPLOAD_PART" - __format: "json" -``` - -You can then recompile it with: - -``` -npm run build -``` - -And it can be started with: - -``` -NODE_ENV=production NODE_CONFIG_DIR=/srv/peertube/config node dist/server.js -``` +In return, this system is a bit harder to configure. +We show how it is still possible to configure Garage with Peertube, allowing you to spread the load and the bandwidth usage on the Garage cluster. ### Create resources in Garage @@ -232,30 +144,31 @@ garage bucket create peertube-playlist Now we allow our key to read and write on these buckets: ``` -garage bucket allow peertube-playlist --read --write --key peertube-key -garage bucket allow peertube-video --read --write --key peertube-key +garage bucket allow peertube-playlists --read --write --owner --key peertube-key +garage bucket allow peertube-videos --read --write --owner --key peertube-key ``` -Finally, we need to expose these buckets publicly to serve their content to users: +We also need to expose these buckets publicly to serve their content to users: ```bash -garage bucket website --allow peertube-playlist -garage bucket website --allow peertube-video +garage bucket website --allow peertube-playlists +garage bucket website --allow peertube-videos ``` -These buckets are now accessible on the web port (by default 3902) with the following URL: `http://<bucket><root_domain>:<web_port>` where the root domain is defined in your configuration file (by default `.web.garage`). So we have currently the following URLs: - * http://peertube-playlist.web.garage:3902 - * http://peertube-video.web.garage:3902 +Finally, we must allow Cross-Origin Resource Sharing (CORS). +CORS are required by your browser to allow requests triggered from the peertube website (eg. peertube.tld) to your bucket's domain (eg. peertube-videos.web.garage.tld) -Make sure you (will) have a corresponding DNS entry for them. - -### Configure a Reverse Proxy to serve CORS +```bash +export CORS='{"CORSRules":[{"AllowedHeaders":["*"],"AllowedMethods":["GET"],"AllowedOrigins":["*"]}]}' +aws --endpoint http://s3.garage.localhost s3api put-bucket-cors --bucket peertube-playlists --cors-configuration $CORS +aws --endpoint http://s3.garage.localhost s3api put-bucket-cors --bucket peertube-videos --cors-configuration $CORS +``` -Now we will configure a reverse proxy in front of Garage. -This is required as we have no other way to serve CORS headers yet. -Check the [Configuring a reverse proxy](/cookbook/reverse_proxy.html) section to know how. +These buckets are now accessible on the web port (by default 3902) with the following URL: `http://<bucket><root_domain>:<web_port>` where the root domain is defined in your configuration file (by default `.web.garage`). So we have currently the following URLs: + * http://peertube-playlists.web.garage:3902 + * http://peertube-videos.web.garage:3902 -Now make sure that your 2 dns entries are pointing to your reverse proxy. +Make sure you (will) have a corresponding DNS entry for them. ### Configure Peertube @@ -268,9 +181,6 @@ object_storage: # Put localhost only if you have a garage instance running on that node endpoint: 'http://localhost:3900' # or "garage.example.com" if you have TLS on port 443 - # This entry has been added by our patch, must be set to true - force_path_style: true - # Garage supports only one region for now, named garage region: 'garage' @@ -287,28 +197,23 @@ object_storage: prefix: '' # You must fill this field to make Peertube use our reverse proxy/website logic - base_url: 'http://peertube-playlist.web.garage' # Example: 'https://mirror.example.com' + base_url: 'http://peertube-playlists.web.garage.localhost' # Example: 'https://mirror.example.com' # Same settings but for webtorrent videos videos: bucket_name: 'peertube-video' prefix: '' # You must fill this field to make Peertube use our reverse proxy/website logic - base_url: 'http://peertube-video.web.garage' + base_url: 'http://peertube-videos.web.garage.localhost' ``` ### That's all Everything must be configured now, simply restart Peertube and try to upload a video. -You must see in your browser console that data are fetched directly from our bucket (through the reverse proxy). - -### Miscellaneous - -*Known bug:* The playback does not start and some 400 Bad Request Errors appear in your browser console and on Garage. -If the description of the error contains HTTP Invalid Range: InvalidRange, the error is due to a buggy ffmpeg version. -You must avoid the 4.4.0 and use either a newer or older version. -*Associated issues:* [#137](https://git.deuxfleurs.fr/Deuxfleurs/garage/issues/137), [#138](https://git.deuxfleurs.fr/Deuxfleurs/garage/issues/138), [#140](https://git.deuxfleurs.fr/Deuxfleurs/garage/issues/140). These issues are non blocking. +Peertube will start by serving the video from its own domain while it is encoding. +Once the encoding is done, the video is uploaded to Garage. +You can now reload the page and see in your browser console that data are fetched directly from your bucket. *External link:* [Peertube Documentation > Remote Storage](https://docs.joinpeertube.org/admin-remote-storage) diff --git a/doc/book/src/connect/publii.png b/doc/book/src/connect/publii.png Binary files differnew file mode 100644 index 00000000..4b175f53 --- /dev/null +++ b/doc/book/src/connect/publii.png diff --git a/doc/book/src/connect/websites.md b/doc/book/src/connect/websites.md index 6f66c8d0..6c3197df 100644 --- a/doc/book/src/connect/websites.md +++ b/doc/book/src/connect/websites.md @@ -3,6 +3,12 @@ Garage is also suitable to host static websites. While they can be deployed with traditional CLI tools, some static website generators have integrated options to ease your workflow. +| Name | Status | Note | +|------|--------|------| +| [Hugo](#hugo) | ✅ | Publishing logic is integrated in the tool | +| [Publii](#publii) | ✅ | Require a correctly configured s3 vhost endpoint | +| [Generic Static Site Generator](#generic-static-site-generator) | ✅ | Works for Jekyll, Zola, Gatsby, Pelican, etc. | + ## Hugo Add to your `config.toml` the following section: @@ -39,39 +45,38 @@ hugo deploy ## Publii -It would require a patch either on Garage or on Publii to make both systems work. +[](./publii.png) -Currently, the proposed workaround is to deploy your website manually: - - On the left menu, click on Server, choose Manual Deployment (the logo looks like a compressed file) - - Set your website URL, keep Output type as "Non-compressed catalog" - - Click on Save changes - - Click on Sync your website (bottom left of the app) - - On the new page, click again on Sync your website - - Click on Get website files - - You need to synchronize the output folder you see in your file explorer, we will use minio client. +Deploying a website to Garage from Publii is natively supported. +First, make sure that your Garage administrator allowed and configured Garage to support vhost access style. +We also suppose that your bucket ("my-bucket") and key is already created and configured. -Be sure that you [configured minio client](cli.html#minio-client-recommended). +Then, from the left menu, click on server. Choose "S3" as the protocol. +In the configuration window, enter: + - Your finale website URL (eg. "http://my-bucket.web.garage.localhost:3902") + - Tick "Use a custom S3 provider" + - Set the S3 endpoint, (eg. "http://s3.garage.localhost:3900") + - Then put your access key (eg. "GK..."), your secret key, and your bucket (eg. "my-bucket") + - And hit the button "Save settings" + +Now, each time you want to publish your website from Publii, just hit the bottom left button "Sync your website"! -Then copy this output folder -```bash -mc mirror --overwrite output garage/my-site -``` -## Generic (eg. Jekyll) +## Generic Static Site Generator Some tools do not support sending to a S3 backend but output a compiled folder on your system. We can then use any CLI tool to upload this content to our S3 target. First, start by [configuring minio client](cli.html#minio-client-recommended). -Then build your website: +Then build your website (example for jekyll): ```bash jekyll build ``` -And copy jekyll's output folder on S3: +And copy its output folder (`_site` for Jekyll) on S3: ```bash mc mirror --overwrite _site garage/my-site |