From e7e164a280dfc1c4adf9d6da6f3b2a9674eca4bd Mon Sep 17 00:00:00 2001 From: Alex Auvolat Date: Fri, 9 Jun 2023 16:23:21 +0200 Subject: Make fsync an option for meta and data --- doc/book/reference-manual/configuration.md | 45 ++++++++++++++++++++++++++++++ 1 file changed, 45 insertions(+) (limited to 'doc/book/reference-manual/configuration.md') diff --git a/doc/book/reference-manual/configuration.md b/doc/book/reference-manual/configuration.md index 38062bab..de253393 100644 --- a/doc/book/reference-manual/configuration.md +++ b/doc/book/reference-manual/configuration.md @@ -10,6 +10,8 @@ Here is an example `garage.toml` configuration file that illustrates all of the ```toml metadata_dir = "/var/lib/garage/meta" data_dir = "/var/lib/garage/data" +metadata_fsync = true +data_fsync = false db_engine = "lmdb" @@ -124,6 +126,49 @@ convert-db -a -i \ Make sure to specify the full database path as presented in the table above, and not just the path to the metadata directory. +### `metadata_fsync` + +Whether to enable synchronous mode for the database engine or not. +This is disabled (`false`) by default. + +This reduces the risk of metadata corruption in case of power failures, +at the cost of a significant drop in write performance, +as Garage will have to pause to sync data to disk much more often +(several times for API calls such as PutObject). + +Using this option reduces the risk of simultaneous metadata corruption on several +cluster nodes, which could lead to data loss. + +If multi-site replication is used, this option is most likely not necessary, as +it is extremely unlikely that two nodes in different locations will have a +power failure at the exact same time. + +(Metadata corruption on a single node is not an issue, the corrupted data file +can always be deleted and reconstructed from the other nodes in the cluster.) + +Here is how this option impacts the different database engines: + +| Database | `metadata_fsync = false` (default) | `metadata_fsync = true` | +|----------|------------------------------------|-------------------------------| +| Sled | default options | *unsupported* | +| Sqlite | `PRAGMA synchronous = OFF` | `PRAGMA synchronous = NORMAL` | +| LMDB | `MDB_NOMETASYNC` + `MDB_NOSYNC` | `MDB_NOMETASYNC` | + +Note that the Sqlite database is always ran in `WAL` mode (`PRAGMA journal_mode = WAL`). + +### `data_fsync` + +Whether to `fsync` data blocks and their containing directory after they are +saved to disk. +This is disabled (`false`) by default. + +This might reduce the risk that a data block is lost in rare +situations such as simultaneous node losing power, +at the cost of a moderate drop in write performance. + +Similarly to `metatada_fsync`, this is likely not necessary +if geographical replication is used. + ### `block_size` Garage splits stored objects in consecutive chunks of size `block_size` -- cgit v1.2.3 From 6595efd82fc849c97b964969b6ff935738e7d24a Mon Sep 17 00:00:00 2001 From: Alex Auvolat Date: Thu, 7 Sep 2023 13:23:02 +0200 Subject: Document multi-hdd support --- doc/book/reference-manual/configuration.md | 13 +++++++++++++ 1 file changed, 13 insertions(+) (limited to 'doc/book/reference-manual/configuration.md') diff --git a/doc/book/reference-manual/configuration.md b/doc/book/reference-manual/configuration.md index 3f6ec091..df1251c2 100644 --- a/doc/book/reference-manual/configuration.md +++ b/doc/book/reference-manual/configuration.md @@ -91,6 +91,19 @@ This folder can be placed on an HDD. The space available for `data_dir` should be counted to determine a node's capacity when [adding it to the cluster layout](@/documentation/cookbook/real-world.md). +Since `v0.9.0`, Garage supports multiple data directories with the following syntax: + +```toml +data_dir = [ + { path = "/path/to/old_data", read_only = true }, + { path = "/path/to/new_hdd1", capacity = "2T" }, + { path = "/path/to/new_hdd2", capacity = "4T" }, +] +``` + +See [the dedicated documentation page](@/documentation/operations/multi-hdd.md) +on how to operate Garage in such a setup. + ### `db_engine` (since `v0.8.0`) By default, Garage uses the Sled embedded database library -- cgit v1.2.3