diff options
Diffstat (limited to 'docs/configuration')
-rw-r--r-- | docs/configuration/auth.md | 1 | ||||
-rw-r--r-- | docs/configuration/cheatsheet.md | 72 | ||||
-rw-r--r-- | docs/configuration/howto_database_config.md | 98 | ||||
-rw-r--r-- | docs/configuration/howto_ejabberd.md | 136 | ||||
-rw-r--r-- | docs/configuration/howto_search_cjk.md | 42 | ||||
-rw-r--r-- | docs/configuration/mrf.md | 23 | ||||
-rw-r--r-- | docs/configuration/optimizing_beam.md | 66 | ||||
-rw-r--r-- | docs/configuration/postgresql.md | 27 | ||||
-rw-r--r-- | docs/configuration/static_dir.md | 5 |
9 files changed, 397 insertions, 73 deletions
diff --git a/docs/configuration/auth.md b/docs/configuration/auth.md new file mode 100644 index 000000000..c80f094e7 --- /dev/null +++ b/docs/configuration/auth.md @@ -0,0 +1 @@ +See `Authentication` section of [the configuration cheatsheet](../configuration/cheatsheet.md#authentication). diff --git a/docs/configuration/cheatsheet.md b/docs/configuration/cheatsheet.md index 2c41ee932..028c5e91d 100644 --- a/docs/configuration/cheatsheet.md +++ b/docs/configuration/cheatsheet.md @@ -45,10 +45,11 @@ To add configuration to your config file, you can copy it from the base config. older software for theses nicknames. * `max_pinned_statuses`: The maximum number of pinned statuses. `0` will disable the feature. * `autofollowed_nicknames`: Set to nicknames of (local) users that every new user should automatically follow. +* `autofollowing_nicknames`: Set to nicknames of (local) users that automatically follows every newly registered user. * `attachment_links`: Set to true to enable automatically adding attachment link text to statuses. * `max_report_comment_size`: The maximum size of the report comment (Default: `1000`). * `safe_dm_mentions`: If set to true, only mentions at the beginning of a post will be used to address people in direct messages. This is to prevent accidental mentioning of people when talking about them (e.g. "@friend hey i really don't like @enemy"). Default: `false`. -* `healthcheck`: If set to true, system data will be shown on ``/api/pleroma/healthcheck``. +* `healthcheck`: If set to true, system data will be shown on ``/api/v1/pleroma/healthcheck``. * `remote_post_retention_days`: The default amount of days to retain remote posts when pruning the database. * `user_bio_length`: A user bio maximum length (default: `5000`). * `user_name_length`: A user name maximum length (default: `100`). @@ -62,6 +63,7 @@ To add configuration to your config file, you can copy it from the base config. * `external_user_synchronization`: Enabling following/followers counters synchronization for external users. * `cleanup_attachments`: Remove attachments along with statuses. Does not affect duplicate files and attachments without status. Enabling this will increase load to database when deleting statuses on larger instances. * `show_reactions`: Let favourites and emoji reactions be viewed through the API (default: `true`). +* `password_reset_token_validity`: The time after which reset tokens aren't accepted anymore, in seconds (default: one day). ## Welcome * `direct_message`: - welcome message sent as a direct message. @@ -219,13 +221,11 @@ config :pleroma, :mrf_user_allowlist, %{ * `total_user_limit`: the number of scheduled activities a user is allowed to create in total (Default: `300`) * `enabled`: whether scheduled activities are sent to the job queue to be executed -## Frontends - ### :frontend_configurations This can be used to configure a keyword list that keeps the configuration data for any kind of frontend. By default, settings for `pleroma_fe` and `masto_fe` are configured. You can find the documentation for `pleroma_fe` configuration into [Pleroma-FE configuration and customization for instance administrators](/frontend/CONFIGURATION/#options). -Frontends can access these settings at `/api/pleroma/frontend_configurations` +Frontends can access these settings at `/api/v1/pleroma/frontend_configurations` To add your own configuration for PleromaFE, use it like this: @@ -321,9 +321,10 @@ This section describe PWA manifest instance-specific values. Currently this opti #### Pleroma.Web.MediaProxy.Invalidation.Script This strategy allow perform external shell script to purge cache. -Urls of attachments pass to script as arguments. +Urls of attachments are passed to the script as arguments. -* `script_path`: path to external script. +* `script_path`: Path to the external script. +* `url_format`: Set to `:htcacheclean` if using Apache's htcacheclean utility. Example: @@ -549,7 +550,7 @@ the source code is here: [kocaptcha](https://github.com/koto-bank/kocaptcha). Th * `uploader`: Which one of the [uploaders](#uploaders) to use. * `filters`: List of [upload filters](#upload-filters) to use. * `link_name`: When enabled Pleroma will add a `name` parameter to the url of the upload, for example `https://instance.tld/media/corndog.png?name=corndog.png`. This is needed to provide the correct filename in Content-Disposition headers when using filters like `Pleroma.Upload.Filter.Dedupe` -* `base_url`: The base URL to access a user-uploaded file. Useful when you want to proxy the media files via another host. +* `base_url`: The base URL to access a user-uploaded file. Useful when you want to host the media files via another domain or are using a 3rd party S3 provider. * `proxy_remote`: If you're using a remote uploader, Pleroma will proxy media requests instead of redirecting to it. * `proxy_opts`: Proxy options, see `Pleroma.ReverseProxy` documentation. * `filename_display_max_length`: Set max length of a filename to display. 0 = no limit. Default: 30. @@ -570,10 +571,7 @@ Don't forget to configure [Ex AWS S3](#ex-aws-s3-settings) * `bucket`: S3 bucket name. * `bucket_namespace`: S3 bucket namespace. -* `public_endpoint`: S3 endpoint that the user finally accesses(ex. "https://s3.dualstack.ap-northeast-1.amazonaws.com") * `truncated_namespace`: If you use S3 compatible service such as Digital Ocean Spaces or CDN, set folder name or "" etc. -For example, when using CDN to S3 virtual host format, set "". -At this time, write CNAME to CDN in public_endpoint. * `streaming_enabled`: Enable streaming uploads, when enabled the file will be sent to the server in chunks as it's being read. This may be unsupported by some providers, try disabling this if you have upload problems. #### Ex AWS S3 settings @@ -850,13 +848,13 @@ config :pleroma, :admin_token, "somerandomtoken" You can then do ```shell -curl "http://localhost:4000/api/pleroma/admin/users/invites?admin_token=somerandomtoken" +curl "http://localhost:4000/api/v1/pleroma/admin/users/invites?admin_token=somerandomtoken" ``` or ```shell -curl -H "X-Admin-Token: somerandomtoken" "http://localhost:4000/api/pleroma/admin/users/invites" +curl -H "X-Admin-Token: somerandomtoken" "http://localhost:4000/api/v1/pleroma/admin/users/invites" ``` Warning: it's discouraged to use this feature because of the associated security risk: static / rarely changed instance-wide token is much weaker compared to email-password pair of a real admin user; consider using HTTP Basic Auth or OAuth-based authentication instead. @@ -895,6 +893,22 @@ Pleroma account will be created with the same name as the LDAP user name. Note, if your LDAP server is an Active Directory server the correct value is commonly `uid: "cn"`, but if you use an OpenLDAP server the value may be `uid: "uid"`. +### :oauth2 (Pleroma as OAuth 2.0 provider settings) + +OAuth 2.0 provider settings: + +* `token_expires_in` - The lifetime in seconds of the access token. +* `issue_new_refresh_token` - Keeps old refresh token or generate new refresh token when to obtain an access token. +* `clean_expired_tokens` - Enable a background job to clean expired oauth tokens. Defaults to `false`. + +OAuth 2.0 provider and related endpoints: + +* `POST /api/v1/apps` creates client app basing on provided params. +* `GET/POST /oauth/authorize` renders/submits authorization form. +* `POST /oauth/token` creates/renews OAuth token. +* `POST /oauth/revoke` revokes provided OAuth token. +* `GET /api/v1/accounts/verify_credentials` (with proper `Authorization` header or `access_token` URI param) returns user info on requester (with `acct` field containing local nickname and `fqn` field containing fully-qualified nickname which could generally be used as email stub for OAuth software that demands email field in identity endpoint response, like Peertube). + ### OAuth consumer mode OAuth consumer mode allows sign in / sign up via external OAuth providers (e.g. Twitter, Facebook, Google, Microsoft, etc.). @@ -967,14 +981,6 @@ config :ueberauth, Ueberauth, ] ``` -### OAuth 2.0 provider - :oauth2 - -Configure OAuth 2 provider capabilities: - -* `token_expires_in` - The lifetime in seconds of the access token. -* `issue_new_refresh_token` - Keeps old refresh token or generate new refresh token when to obtain an access token. -* `clean_expired_tokens` - Enable a background job to clean expired oauth tokens. Defaults to `false`. - ## Link parsing ### :uri_schemes @@ -1067,6 +1073,20 @@ Control favicons for instances. * `enabled`: Allow/disallow displaying and getting instances favicons +## Pleroma.User.Backup + +!!! note + Requires enabled email + +* `:purge_after_days` an integer, remove backup achives after N days. +* `:limit_days` an integer, limit user to export not more often than once per N days. +* `:dir` a string with a path to backup temporary directory or `nil` to let Pleroma choose temporary directory in the following order: + 1. the directory named by the TMPDIR environment variable + 2. the directory named by the TEMP environment variable + 3. the directory named by the TMP environment variable + 4. C:\TMP on Windows or /tmp on Unix-like operating systems + 5. as a last resort, the current working directory + ## Frontend management Frontends in Pleroma are swappable - you can specify which one to use here. @@ -1099,3 +1119,15 @@ Settings to enable and configure expiration for ephemeral activities * `:enabled` - enables ephemeral activities creation * `:min_lifetime` - minimum lifetime for ephemeral activities (in seconds). Default: 10 minutes. + +## ConcurrentLimiter + +Settings to restrict concurrently running jobs. Jobs which can be configured: + +* `Pleroma.Web.RichMedia.Helpers` - generating link previews of URLs in activities +* `Pleroma.Web.ActivityPub.MRF.MediaProxyWarmingPolicy` - warming remote media cache via MediaProxyWarmingPolicy + +Each job has these settings: + +* `:max_running` - max concurrently runnings jobs +* `:max_waiting` - max waiting jobs diff --git a/docs/configuration/howto_database_config.md b/docs/configuration/howto_database_config.md index 9ed4d6cdd..ae1462f9b 100644 --- a/docs/configuration/howto_database_config.md +++ b/docs/configuration/howto_database_config.md @@ -5,50 +5,37 @@ The configuration of Pleroma has traditionally been managed with a config file, ## Migration to database config -1. Run the mix task to migrate to the database. You'll receive some debugging output and a few messages informing you of what happened. +1. Run the mix task to migrate to the database. **Source:** - + ``` $ mix pleroma.config migrate_to_db ``` - + or - + **OTP:** - + *Note: OTP users need Pleroma to be running for `pleroma_ctl` commands to work* - + ``` $ ./bin/pleroma_ctl config migrate_to_db ``` ``` - 10:04:34.155 [debug] QUERY OK source="config" db=1.6ms decode=2.0ms queue=33.5ms idle=0.0ms - SELECT c0."id", c0."key", c0."group", c0."value", c0."inserted_at", c0."updated_at" FROM "config" AS c0 [] Migrating settings from file: /home/pleroma/config/dev.secret.exs - - 10:04:34.240 [debug] QUERY OK db=4.5ms queue=0.3ms idle=92.2ms - TRUNCATE config; [] - - 10:04:34.244 [debug] QUERY OK db=2.8ms queue=0.3ms idle=97.2ms - ALTER SEQUENCE config_id_seq RESTART; [] - - 10:04:34.256 [debug] QUERY OK source="config" db=0.8ms queue=1.4ms idle=109.8ms - SELECT c0."id", c0."key", c0."group", c0."value", c0."inserted_at", c0."updated_at" FROM "config" AS c0 WHERE ((c0."group" = $1) AND (c0."key" = $2)) [":pleroma", ":instance"] - - 10:04:34.292 [debug] QUERY OK db=2.6ms queue=1.7ms idle=137.7ms - INSERT INTO "config" ("group","key","value","inserted_at","updated_at") VALUES ($1,$2,$3,$4,$5) RETURNING "id" [":pleroma", ":instance", <<131, 108, 0, 0, 0, 1, 104, 2, 100, 0, 4, 110, 97, 109, 101, 109, 0, 0, 0, 7, 66, 108, 101, 114, 111, 109, 97, 106>>, ~N[2020-07-12 15:04:34], ~N[2020-07-12 15:04:34]] + Settings for key instance migrated. Settings for group :pleroma migrated. ``` - + 2. It is recommended to backup your config file now. ``` cp config/dev.secret.exs config/dev.secret.exs.orig ``` - + 3. Edit your Pleroma config to enable database configuration: ``` @@ -76,17 +63,17 @@ The configuration of Pleroma has traditionally been managed with a config file, config :pleroma, Pleroma.Web.Endpoint, url: [host: "cool.pleroma.site", scheme: "https", port: 443] - + config :pleroma, Pleroma.Repo, adapter: Ecto.Adapters.Postgres, username: "pleroma", password: "MySecretPassword", database: "pleroma_prod", hostname: "localhost" - + config :pleroma, configurable_from_database: true ``` - + 5. Restart your instance and you can now access the Settings tab in AdminFE. @@ -95,15 +82,15 @@ The configuration of Pleroma has traditionally been managed with a config file, 1. Run the mix task to migrate back from the database. You'll receive some debugging output and a few messages informing you of what happened. **Source:** - + ``` $ mix pleroma.config migrate_from_db ``` - + or - + **OTP:** - + ``` $ ./bin/pleroma_ctl config migrate_from_db ``` @@ -111,7 +98,7 @@ The configuration of Pleroma has traditionally been managed with a config file, ``` 10:26:30.593 [debug] QUERY OK source="config" db=9.8ms decode=1.2ms queue=26.0ms idle=0.0ms SELECT c0."id", c0."key", c0."group", c0."value", c0."inserted_at", c0."updated_at" FROM "config" AS c0 [] - + 10:26:30.659 [debug] QUERY OK source="config" db=1.1ms idle=80.7ms SELECT c0."id", c0."key", c0."group", c0."value", c0."inserted_at", c0."updated_at" FROM "config" AS c0 [] Database configuration settings have been saved to config/dev.exported_from_db.secret.exs @@ -124,30 +111,45 @@ The configuration of Pleroma has traditionally been managed with a config file, ## Debugging ### Clearing database config -You can clear the database config by truncating the `config` table in the database. e.g., +You can clear the database config with the following command: + + **Source:** -``` -psql -d pleroma_dev -pleroma_dev=# TRUNCATE config; -TRUNCATE TABLE -``` + ``` + $ mix pleroma.config reset + ``` + + or + + **OTP:** + + ``` + $ ./bin/pleroma_ctl config reset + ``` Additionally, every time you migrate the configuration to the database the config table is automatically truncated to ensure a clean migration. ### Manually removing a setting If you encounter a situation where the server cannot run properly because of an invalid setting in the database and this is preventing you from accessing AdminFE, you can manually remove the offending setting if you know which one it is. -e.g., here is an example showing a minimal configuration in the database. Only the `config :pleroma, :instance` settings are in the table: - -``` -psql -d pleroma_dev -pleroma_dev=# select * from config; - id | key | value | inserted_at | updated_at | group -----+-----------+------------------------------------------------------------+---------------------+---------------------+---------- - 1 | :instance | \x836c0000000168026400046e616d656d00000007426c65726f6d616a | 2020-07-12 15:33:29 | 2020-07-12 15:33:29 | :pleroma -(1 row) -pleroma_dev=# delete from config where key = ':instance' and group = ':pleroma'; -DELETE 1 -``` +e.g., here is an example showing a the removal of the `config :pleroma, :instance` settings: + + **Source:** + + ``` + $ mix pleroma.config delete pleroma instance + Are you sure you want to continue? [n] y + config :pleroma, :instance deleted from the ConfigDB. + ``` + + or + + **OTP:** + + ``` + $ ./bin/pleroma_ctl config delete pleroma instance + Are you sure you want to continue? [n] y + config :pleroma, :instance deleted from the ConfigDB. + ``` Now the `config :pleroma, :instance` settings have been removed from the database. diff --git a/docs/configuration/howto_ejabberd.md b/docs/configuration/howto_ejabberd.md new file mode 100644 index 000000000..520a0acbc --- /dev/null +++ b/docs/configuration/howto_ejabberd.md @@ -0,0 +1,136 @@ +# Configuring Ejabberd (XMPP Server) to use Pleroma for authentication + +If you want to give your Pleroma users an XMPP (chat) account, you can configure [Ejabberd](https://github.com/processone/ejabberd) to use your Pleroma server for user authentication, automatically giving every local user an XMPP account. + +In general, you just have to follow the configuration described at [https://docs.ejabberd.im/admin/configuration/authentication/#external-script](https://docs.ejabberd.im/admin/configuration/authentication/#external-script). Please read this section carefully. + +Copy the script below to suitable path on your system and set owner and permissions. Also do not forget adjusting `PLEROMA_HOST` and `PLEROMA_PORT`, if necessary. + +```bash +cp pleroma_ejabberd_auth.py /etc/ejabberd/pleroma_ejabberd_auth.py +chown ejabberd /etc/ejabberd/pleroma_ejabberd_auth.py +chmod 700 /etc/ejabberd/pleroma_ejabberd_auth.py +``` + +Set external auth params in ejabberd.yaml file: + +```bash +auth_method: [external] +extauth_program: "python3 /etc/ejabberd/pleroma_ejabberd_auth.py" +extauth_instances: 3 +auth_use_cache: false +``` + +Restart / reload your ejabberd service. + +After restarting your Ejabberd server, your users should now be able to connect with their Pleroma credentials. + + +```python +import sys +import struct +import http.client +from base64 import b64encode +import logging + + +PLEROMA_HOST = "127.0.0.1" +PLEROMA_PORT = "4000" +AUTH_ENDPOINT = "/api/v1/accounts/verify_credentials" +USER_ENDPOINT = "/api/v1/accounts" +LOGFILE = "/var/log/ejabberd/pleroma_auth.log" + +logging.basicConfig(filename=LOGFILE, level=logging.INFO) + + +# Pleroma functions +def create_connection(): + return http.client.HTTPConnection(PLEROMA_HOST, PLEROMA_PORT) + + +def verify_credentials(user: str, password: str) -> bool: + user_pass_b64 = b64encode("{}:{}".format( + user, password).encode('utf-8')).decode("ascii") + params = {} + headers = { + "Authorization": "Basic {}".format(user_pass_b64) + } + + try: + conn = create_connection() + conn.request("GET", AUTH_ENDPOINT, params, headers) + + response = conn.getresponse() + if response.status == 200: + return True + + return False + except Exception as e: + logging.info("Can not connect: %s", str(e)) + return False + + +def does_user_exist(user: str) -> bool: + conn = create_connection() + conn.request("GET", "{}/{}".format(USER_ENDPOINT, user)) + + response = conn.getresponse() + if response.status == 200: + return True + + return False + + +def auth(username: str, server: str, password: str) -> bool: + return verify_credentials(username, password) + + +def isuser(username, server): + return does_user_exist(username) + + +def read(): + (pkt_size,) = struct.unpack('>H', bytes(sys.stdin.read(2), encoding='utf8')) + pkt = sys.stdin.read(pkt_size) + cmd = pkt.split(':')[0] + if cmd == 'auth': + username, server, password = pkt.split(':', 3)[1:] + write(auth(username, server, password)) + elif cmd == 'isuser': + username, server = pkt.split(':', 2)[1:] + write(isuser(username, server)) + elif cmd == 'setpass': + # u, s, p = pkt.split(':', 3)[1:] + write(False) + elif cmd == 'tryregister': + # u, s, p = pkt.split(':', 3)[1:] + write(False) + elif cmd == 'removeuser': + # u, s = pkt.split(':', 2)[1:] + write(False) + elif cmd == 'removeuser3': + # u, s, p = pkt.split(':', 3)[1:] + write(False) + else: + write(False) + + +def write(result): + if result: + sys.stdout.write('\x00\x02\x00\x01') + else: + sys.stdout.write('\x00\x02\x00\x00') + sys.stdout.flush() + + +if __name__ == "__main__": + logging.info("Starting pleroma ejabberd auth daemon...") + while True: + try: + read() + except Exception as e: + logging.info( + "Error while processing data from ejabberd %s", str(e)) + pass + +```
\ No newline at end of file diff --git a/docs/configuration/howto_search_cjk.md b/docs/configuration/howto_search_cjk.md new file mode 100644 index 000000000..d3ce28077 --- /dev/null +++ b/docs/configuration/howto_search_cjk.md @@ -0,0 +1,42 @@ +# How to enable text search for Chinese, Japanese and Korean + +Pleroma's full text search feature is powered by PostgreSQL's native [text search](https://www.postgresql.org/docs/current/textsearch.html), it works well out of box for most of languages, but needs extra configurations for some asian languages like Chinese, Japanese and Korean (CJK). + + +## Setup and test the new search config + +In most cases, you would need an extension installed to support parsing CJK text. Here are a few extension you may choose from, or you are more than welcome to share additional ones you found working for you with the rest of Pleroma community. + + * [a generic n-gram parser](https://github.com/huangjimmy/pg_cjk_parser) supports Simplifed/Traditional Chinese, Japanese, and Korean + * [a Korean parser](https://github.com/i0seph/textsearch_ko) based on mecab + * [a Japanese parser](https://www.amris.co.jp/tsja/index.html) based on mecab + * [zhparser](https://github.com/amutu/zhparser/) is a PostgreSQL extension base on the Simple Chinese Word Segmentation(SCWS) + * [another Chinese parser](https://github.com/jaiminpan/pg_jieba) based on Jieba Chinese Word Segmentation + +Once you have the new search config , make sure you test it with the `pleroma` user in PostgreSQL (change `YOUR.CONFIG` to your real configuration name) +``` +SELECT ts_debug('YOUR.CONFIG', '安装和配置Nginx, ElixirとErlangをインストールします'); +``` +Check output of the query, and see if it matches your expectation. + + +## Update text search config and index in database + +=== "OTP" + + ```sh + ./bin/pleroma_ctl database set_text_search_config YOUR.CONFIG + ``` + +=== "From Source" + + ```sh + mix pleroma.database set_text_search_config YOUR.CONFIG + ``` + +Note: index update may take a while. + +## Restart database connection +Since some changes above will only apply with a new database connection, you will have to restart either Pleroma or PostgreSQL process, or use `pg_terminate_backend` SQL command without restarting either. + +Now the search results of statuses should be much more friendly for your language of choice, the results for searching users and tags were not changed, as the default parsing/matching should work for most cases. diff --git a/docs/configuration/mrf.md b/docs/configuration/mrf.md index 31c66e098..9e8c0a2d7 100644 --- a/docs/configuration/mrf.md +++ b/docs/configuration/mrf.md @@ -133,3 +133,26 @@ config :pleroma, :mrf, ``` Please note that the Pleroma developers consider custom MRF policy modules to fall under the purview of the AGPL. As such, you are obligated to release the sources to your custom MRF policy modules upon request. + +### MRF policies descriptions + +If MRF policy depends on config, it can be added into MRF tab to adminFE by adding `config_description/0` method, which returns a map with a specific structure. See existing MRF's like `lib/pleroma/web/activity_pub/mrf/activity_expiration_policy.ex` for examples. Note that more complex inputs, like tuples or maps, may need extra changes in the adminFE and just adding it to `config_description/0` may not be enough to get these inputs working from the adminFE. + +Example: + +```elixir +%{ + key: :mrf_activity_expiration, + related_policy: "Pleroma.Web.ActivityPub.MRF.ActivityExpirationPolicy", + label: "MRF Activity Expiration Policy", + description: "Adds automatic expiration to all local activities", + children: [ + %{ + key: :days, + type: :integer, + description: "Default global expiration time for all local activities (in days)", + suggestions: [90, 365] + } + ] + } +``` diff --git a/docs/configuration/optimizing_beam.md b/docs/configuration/optimizing_beam.md new file mode 100644 index 000000000..e336bd36c --- /dev/null +++ b/docs/configuration/optimizing_beam.md @@ -0,0 +1,66 @@ +# Optimizing the BEAM + +Pleroma is built upon the Erlang/OTP VM known as BEAM. The BEAM VM is highly optimized for latency, but this has drawbacks in environments without dedicated hardware. One of the tricks used by the BEAM VM is [busy waiting](https://en.wikipedia.org/wiki/Busy_waiting). This allows the application to pretend to be busy working so the OS kernel does not pause the application process and switch to another process waiting for the CPU to execute its workload. It does this by spinning for a period of time which inflates the apparent CPU usage of the application so it is immediately ready to execute another task. This can be observed with utilities like **top(1)** which will show consistently high CPU usage for the process. Switching between procesess is a rather expensive operation and also clears CPU caches further affecting latency and performance. The goal of busy waiting is to avoid this penalty. + +This strategy is very successful in making a performant and responsive application, but is not desirable on Virtual Machines or hardware with few CPU cores. Pleroma instances are often deployed on the same server as the required PostgreSQL database which can lead to situations where the Pleroma application is holding the CPU in a busy-wait loop and as a result the database cannot process requests in a timely manner. The fewer CPUs available, the more this problem is exacerbated. The latency is further amplified by the OS being installed on a Virtual Machine as the Hypervisor uses CPU time-slicing to pause the entire OS and switch between other tasks. + +More adventurous admins can be creative with CPU affinity (e.g., *taskset* for Linux and *cpuset* on FreeBSD) to pin processes to specific CPUs and eliminate much of this contention. The most important advice is to run as few processes as possible on your server to achieve the best performance. Even idle background processes can occasionally create [software interrupts](https://en.wikipedia.org/wiki/Interrupt) and take attention away from the executing process creating latency spikes and invalidation of the CPU caches as they must be cleared when switching between processes for security. + +Please only change these settings if you are experiencing issues or really know what you are doing. In general, there's no need to change these settings. + +## VPS Provider Recommendations + +### Good + +* Hetzner Cloud + +### Bad + +* AWS (known to use burst scheduling) + + +## Example configurations + +Tuning the BEAM requires you provide a config file normally called [vm.args](http://erlang.org/doc/man/erl.html#emulator-flags). If you are using systemd to manage the service you can modify the unit file as such: + +`ExecStart=/usr/bin/elixir --erl '-args_file /opt/pleroma/config/vm.args' -S /usr/bin/mix phx.server` + +Check your OS documentation to adopt a similar strategy on other platforms. + +### Virtual Machine and/or few CPU cores + +Disable the busy-waiting. This should generally only be done if you're on a platform that does burst scheduling, like AWS. + +**vm.args:** + +``` ++sbwt none ++sbwtdcpu none ++sbwtdio none +``` + +### Dedicated Hardware + +Enable more busy waiting, increase the internal maximum limit of BEAM processes and ports. You can use this if you run on dedicated hardware, but it is not necessary. + +**vm.args:** + +``` ++P 16777216 ++Q 16777216 ++K true ++A 128 ++sbt db ++sbwt very_long ++swt very_low ++sub true ++Mulmbcs 32767 ++Mumbcgs 1 ++Musmbcs 2047 +``` + +## Additional Reading + +* [WhatsApp: Scaling to Millions of Simultaneous Connections](https://www.erlang-factory.com/upload/presentations/558/efsf2012-whatsapp-scaling.pdf) +* [Preemptive Scheduling and Spinlocks](https://www.uio.no/studier/emner/matnat/ifi/nedlagte-emner/INF3150/h03/annet/slides/preemptive.pdf) +* [The Curious Case of BEAM CPU Usage](https://stressgrid.com/blog/beam_cpu_usage/) diff --git a/docs/configuration/postgresql.md b/docs/configuration/postgresql.md index 6983fb459..e251eb83b 100644 --- a/docs/configuration/postgresql.md +++ b/docs/configuration/postgresql.md @@ -1,10 +1,28 @@ -# Optimizing your PostgreSQL performance +# Optimizing PostgreSQL performance -Pleroma performance depends to a large extent on good database performance. The default PostgreSQL settings are mostly fine, but often you can get better performance by changing a few settings. +Pleroma performance is largely dependent on performance of the underlying database. Better performance can be achieved by adjusting a few settings. -You can use [PGTune](https://pgtune.leopard.in.ua) to get recommendations for your setup. If you do, set the "Number of Connections" field to 20, as Pleroma will only use 10 concurrent connections anyway. If you don't, it will give you advice that might even hurt your performance. +## PGTune -We also recommend not using the "Network Storage" option. +[PgTune](https://pgtune.leopard.in.ua) can be used to get recommended settings. Be sure to set "Number of Connections" to 20, otherwise it might produce settings hurtful to database performance. It is also recommended to not use "Network Storage" option. + +## Disable generic query plans + +When PostgreSQL receives a query, it decides on a strategy for searching the requested data, this is called a query plan. The query planner has two modes: generic and custom. Generic makes a plan for all queries of the same shape, ignoring the parameters, which is then cached and reused. Custom, on the contrary, generates a unique query plan based on query parameters. + +By default PostgreSQL has an algorithm to decide which mode is more efficient for particular query, however this algorithm has been observed to be wrong on some of the queries Pleroma sends, leading to serious performance loss. Therefore, it is recommended to disable generic mode. + + +Pleroma already avoids generic query plans by default, however the method it uses is not the most efficient because it needs to be compatible with all supported PostgreSQL versions. For PostgreSQL 12 and higher additional performance can be gained by adding the following to Pleroma configuration: +```elixir +config :pleroma, Pleroma.Repo, + prepare: :named, + parameters: [ + plan_cache_mode: "force_custom_plan" + ] +``` + +A more detailed explaination of the issue can be found at <https://blog.soykaf.com/post/postgresql-elixir-troubles/>. ## Example configurations @@ -28,4 +46,3 @@ max_worker_processes = 2 max_parallel_workers_per_gather = 1 max_parallel_workers = 2 ``` - diff --git a/docs/configuration/static_dir.md b/docs/configuration/static_dir.md index 8ac07b725..a294bb604 100644 --- a/docs/configuration/static_dir.md +++ b/docs/configuration/static_dir.md @@ -88,3 +88,8 @@ config :pleroma, :frontend_configurations, Note the extra `static` folder for the terms-of-service.html Terms of Service will be shown to all users on the registration page. It's the best place where to write down the rules for your instance. You can modify the rules by adding and changing `$static_dir/static/terms-of-service.html`. + + +## Styling rendered pages + +To overwrite the CSS stylesheet of the OAuth form and other static pages, you can upload your own CSS file to `instance/static/static.css`. This will completely replace the CSS used by those pages, so it might be a good idea to copy the one from `priv/static/instance/static.css` and make your changes. |