diff options
Diffstat (limited to 'docs/configuration')
| -rw-r--r-- | docs/configuration/auth.md | 1 | ||||
| -rw-r--r-- | docs/configuration/cheatsheet.md | 145 | ||||
| -rw-r--r-- | docs/configuration/how_to_serve_another_domain_for_webfinger.md | 62 | ||||
| -rw-r--r-- | docs/configuration/howto_database_config.md | 2 | ||||
| -rw-r--r-- | docs/configuration/howto_search_cjk.md | 42 | ||||
| -rw-r--r-- | docs/configuration/mrf.md | 37 | ||||
| -rw-r--r-- | docs/configuration/postgresql.md | 27 |
7 files changed, 255 insertions, 61 deletions
diff --git a/docs/configuration/auth.md b/docs/configuration/auth.md new file mode 100644 index 000000000..c80f094e7 --- /dev/null +++ b/docs/configuration/auth.md @@ -0,0 +1 @@ +See `Authentication` section of [the configuration cheatsheet](../configuration/cheatsheet.md#authentication). diff --git a/docs/configuration/cheatsheet.md b/docs/configuration/cheatsheet.md index 85551362c..314fdbbab 100644 --- a/docs/configuration/cheatsheet.md +++ b/docs/configuration/cheatsheet.md @@ -8,18 +8,19 @@ For from source installations Pleroma configuration works by first importing the To add configuration to your config file, you can copy it from the base config. The latest version of it can be viewed [here](https://git.pleroma.social/pleroma/pleroma/blob/develop/config/config.exs). You can also use this file if you don't know how an option is supposed to be formatted. -## :chat +## :shout -* `enabled` - Enables the backend chat. Defaults to `true`. +* `enabled` - Enables the backend Shoutbox chat feature. Defaults to `true`. +* `limit` - Shout character limit. Defaults to `5_000` ## :instance * `name`: The instance’s name. * `email`: Email used to reach an Administrator/Moderator of the instance. * `notify_email`: Email used for notifications. * `description`: The instance’s description, can be seen in nodeinfo and ``/api/v1/instance``. +* `short_description`: Shorter version of instance description, can be seen on ``/api/v1/instance``. * `limit`: Posts character limit (CW/Subject included in the counter). * `description_limit`: The character limit for image descriptions. -* `chat_limit`: Character limit of the instance chat messages. * `remote_limit`: Hard character limit beyond which remote posts will be dropped. * `upload_limit`: File size limit of uploads (except for avatar, background, banner). * `avatar_upload_limit`: File size limit of user’s profile avatars. @@ -37,9 +38,9 @@ To add configuration to your config file, you can copy it from the base config. * `federating`: Enable federation with other instances. * `federation_incoming_replies_max_depth`: Max. depth of reply-to activities fetching on incoming federation, to prevent out-of-memory situations while fetching very long threads. If set to `nil`, threads of any depth will be fetched. Lower this value if you experience out-of-memory crashes. * `federation_reachability_timeout_days`: Timeout (in days) of each external federation target being unreachable prior to pausing federating to it. -* `allow_relay`: Enable Pleroma’s Relay, which makes it possible to follow a whole instance. +* `allow_relay`: Permits remote instances to subscribe to all public posts of your instance. This may increase the visibility of your instance. * `public`: Makes the client API in authenticated mode-only except for user-profiles. Useful for disabling the Local Timeline and The Whole Known Network. Note that there is a dependent setting restricting or allowing unauthenticated access to specific resources, see `restrict_unauthenticated` for more details. -* `quarantined_instances`: List of ActivityPub instances where private (DMs, followers-only) activities will not be send. +* `quarantined_instances`: ActivityPub instances where private (DMs, followers-only) activities will not be send. * `allowed_post_formats`: MIME-type list of formats allowed to be posted (transformed into HTML). * `extended_nickname_format`: Set to `true` to use extended local nicknames format (allows underscores/dashes). This will break federation with older software for theses nicknames. @@ -48,8 +49,9 @@ To add configuration to your config file, you can copy it from the base config. * `autofollowing_nicknames`: Set to nicknames of (local) users that automatically follows every newly registered user. * `attachment_links`: Set to true to enable automatically adding attachment link text to statuses. * `max_report_comment_size`: The maximum size of the report comment (Default: `1000`). +* `report_strip_status`: Strip associated statuses in reports to ids when closed/resolved, otherwise keep a copy. * `safe_dm_mentions`: If set to true, only mentions at the beginning of a post will be used to address people in direct messages. This is to prevent accidental mentioning of people when talking about them (e.g. "@friend hey i really don't like @enemy"). Default: `false`. -* `healthcheck`: If set to true, system data will be shown on ``/api/pleroma/healthcheck``. +* `healthcheck`: If set to true, system data will be shown on ``/api/v1/pleroma/healthcheck``. * `remote_post_retention_days`: The default amount of days to retain remote posts when pruning the database. * `user_bio_length`: A user bio maximum length (default: `5000`). * `user_name_length`: A user name maximum length (default: `100`). @@ -65,6 +67,13 @@ To add configuration to your config file, you can copy it from the base config. * `show_reactions`: Let favourites and emoji reactions be viewed through the API (default: `true`). * `password_reset_token_validity`: The time after which reset tokens aren't accepted anymore, in seconds (default: one day). +## :database +* `improved_hashtag_timeline`: Setting to force toggle / force disable improved hashtags timeline. `:enabled` forces hashtags to be fetched from `hashtags` table for hashtags timeline. `:disabled` forces object-embedded hashtags to be used (slower). Keep it `:auto` for automatic behaviour (it is auto-set to `:enabled` [unless overridden] when HashtagsTableMigrator completes). + +## Background migrations +* `populate_hashtags_table/sleep_interval_ms`: Sleep interval between each chunk of processed records in order to decrease the load on the system (defaults to 0 and should be keep default on most instances). +* `populate_hashtags_table/fault_rate_allowance`: Max rate of failed objects to actually processed objects in order to enable the feature (any value from 0.0 which tolerates no errors to 1.0 which will enable the feature even if hashtags transfer failed for all records). + ## Welcome * `direct_message`: - welcome message sent as a direct message. * `enabled`: Enables the send a direct message to a newly registered user. Defaults to `false`. @@ -117,6 +126,10 @@ To add configuration to your config file, you can copy it from the base config. * `Pleroma.Web.ActivityPub.MRF.ObjectAgePolicy`: Rejects or delists posts based on their age when received. (See [`:mrf_object_age`](#mrf_object_age)). * `Pleroma.Web.ActivityPub.MRF.ActivityExpirationPolicy`: Sets a default expiration on all posts made by users of the local instance. Requires `Pleroma.Workers.PurgeExpiredActivity` to be enabled for processing the scheduled delections. * `Pleroma.Web.ActivityPub.MRF.ForceBotUnlistedPolicy`: Makes all bot posts to disappear from public timelines. + * `Pleroma.Web.ActivityPub.MRF.FollowBotPolicy`: Automatically follows newly discovered users from the specified bot account. Local accounts, locked accounts, and users with "#nobot" in their bio are respected and excluded from being followed. + * `Pleroma.Web.ActivityPub.MRF.AntiFollowbotPolicy`: Drops follow requests from followbots. Users can still allow bots to follow them by first following the bot. + * `Pleroma.Web.ActivityPub.MRF.KeywordPolicy`: Rejects or removes from the federated timeline or replaces keywords. (See [`:mrf_keyword`](#mrf_keyword)). + * `Pleroma.Web.ActivityPub.MRF.ForceMentionsInContent`: Forces every mentioned user to be reflected in the post content. * `transparency`: Make the content of your Message Rewrite Facility settings public (via nodeinfo). * `transparency_exclusions`: Exclude specific instance names from MRF transparency. The use of the exclusions feature will be disclosed in nodeinfo as a boolean value. @@ -127,15 +140,16 @@ To add configuration to your config file, you can copy it from the base config. Configuring MRF policies is not enough for them to take effect. You have to enable them by specifying their module in `policies` under [:mrf](#mrf) section. #### :mrf_simple -* `media_removal`: List of instances to remove media from. -* `media_nsfw`: List of instances to put media as NSFW(sensitive) from. -* `federated_timeline_removal`: List of instances to remove from Federated (aka The Whole Known Network) Timeline. -* `reject`: List of instances to reject any activities from. -* `accept`: List of instances to accept any activities from. -* `followers_only`: List of instances to decrease post visibility to only the followers, including for DM mentions. -* `report_removal`: List of instances to reject reports from. -* `avatar_removal`: List of instances to strip avatars from. -* `banner_removal`: List of instances to strip banners from. +* `media_removal`: List of instances to strip media attachments from and the reason for doing so. +* `media_nsfw`: List of instances to tag all media as NSFW (sensitive) from and the reason for doing so. +* `federated_timeline_removal`: List of instances to remove from the Federated Timeline (aka The Whole Known Network) and the reason for doing so. +* `reject`: List of instances to reject activities (except deletes) from and the reason for doing so. +* `accept`: List of instances to only accept activities (except deletes) from and the reason for doing so. +* `followers_only`: Force posts from the given instances to be visible by followers only and the reason for doing so. +* `report_removal`: List of instances to reject reports from and the reason for doing so. +* `avatar_removal`: List of instances to strip avatars from and the reason for doing so. +* `banner_removal`: List of instances to strip banners from and the reason for doing so. +* `reject_deletes`: List of instances to reject deletions from and the reason for doing so. #### :mrf_subchain This policy processes messages through an alternate pipeline when a given message matches certain criteria. @@ -191,7 +205,7 @@ config :pleroma, :mrf_user_allowlist, %{ e.g., A value of 900 results in any post with a timestamp older than 15 minutes will be acted upon. * `actions`: A list of actions to apply to the post: * `:delist` removes the post from public timelines - * `:strip_followers` removes followers from the ActivityPub recipient list, ensuring they won't be delivered to home timelines + * `:strip_followers` removes followers from the ActivityPub recipient list, ensuring they won't be delivered to home timelines, additionally for followers-only it degrades to a direct message * `:reject` rejects the message entirely #### :mrf_steal_emoji @@ -203,9 +217,25 @@ config :pleroma, :mrf_user_allowlist, %{ * `days`: Default global expiration time for all local Create activities (in days) +#### :mrf_hashtag + +* `sensitive`: List of hashtags to mark activities as sensitive (default: `nsfw`) +* `federated_timeline_removal`: List of hashtags to remove activities from the federated timeline (aka TWNK) +* `reject`: List of hashtags to reject activities from + +Notes: +- The hashtags in the configuration do not have a leading `#`. +- This MRF Policy is always enabled, if you want to disable it you have to set empty lists + +#### :mrf_follow_bot + +* `follower_nickname`: The name of the bot account to use for following newly discovered users. Using `followbot` or similar is strongly suggested. + + ### :activitypub * `unfollow_blocked`: Whether blocks result in people getting unfollowed * `outgoing_blocks`: Whether to federate blocks to other instances +* `blockers_visible`: Whether a user can see the posts of users who blocked them * `deny_follow_blocked`: Whether to disallow following an account that has blocked the user in question * `sign_object_fetches`: Sign object fetches with HTTP signatures * `authorized_fetch_mode`: Require HTTP signatures for AP fetches @@ -223,9 +253,9 @@ config :pleroma, :mrf_user_allowlist, %{ ### :frontend_configurations -This can be used to configure a keyword list that keeps the configuration data for any kind of frontend. By default, settings for `pleroma_fe` and `masto_fe` are configured. You can find the documentation for `pleroma_fe` configuration into [Pleroma-FE configuration and customization for instance administrators](/frontend/CONFIGURATION/#options). +This can be used to configure a keyword list that keeps the configuration data for any kind of frontend. By default, settings for `pleroma_fe` are configured. You can find the documentation for `pleroma_fe` configuration into [Pleroma-FE configuration and customization for instance administrators](/frontend/CONFIGURATION/#options). -Frontends can access these settings at `/api/pleroma/frontend_configurations` +Frontends can access these settings at `/api/v1/pleroma/frontend_configurations` To add your own configuration for PleromaFE, use it like this: @@ -234,10 +264,7 @@ config :pleroma, :frontend_configurations, pleroma_fe: %{ theme: "pleroma-dark", # ... see /priv/static/static/config.json for the available keys. -}, - masto_fe: %{ - showInstanceSpecificPanel: true - } +} ``` These settings **need to be complete**, they will override the defaults. @@ -321,9 +348,10 @@ This section describe PWA manifest instance-specific values. Currently this opti #### Pleroma.Web.MediaProxy.Invalidation.Script This strategy allow perform external shell script to purge cache. -Urls of attachments pass to script as arguments. +Urls of attachments are passed to the script as arguments. -* `script_path`: path to external script. +* `script_path`: Path to the external script. +* `url_format`: Set to `:htcacheclean` if using Apache's htcacheclean utility. Example: @@ -549,7 +577,7 @@ the source code is here: [kocaptcha](https://github.com/koto-bank/kocaptcha). Th * `uploader`: Which one of the [uploaders](#uploaders) to use. * `filters`: List of [upload filters](#upload-filters) to use. * `link_name`: When enabled Pleroma will add a `name` parameter to the url of the upload, for example `https://instance.tld/media/corndog.png?name=corndog.png`. This is needed to provide the correct filename in Content-Disposition headers when using filters like `Pleroma.Upload.Filter.Dedupe` -* `base_url`: The base URL to access a user-uploaded file. Useful when you want to proxy the media files via another host. +* `base_url`: The base URL to access a user-uploaded file. Useful when you want to host the media files via another domain or are using a 3rd party S3 provider. * `proxy_remote`: If you're using a remote uploader, Pleroma will proxy media requests instead of redirecting to it. * `proxy_opts`: Proxy options, see `Pleroma.ReverseProxy` documentation. * `filename_display_max_length`: Set max length of a filename to display. 0 = no limit. Default: 30. @@ -570,10 +598,7 @@ Don't forget to configure [Ex AWS S3](#ex-aws-s3-settings) * `bucket`: S3 bucket name. * `bucket_namespace`: S3 bucket namespace. -* `public_endpoint`: S3 endpoint that the user finally accesses(ex. "https://s3.dualstack.ap-northeast-1.amazonaws.com") * `truncated_namespace`: If you use S3 compatible service such as Digital Ocean Spaces or CDN, set folder name or "" etc. -For example, when using CDN to S3 virtual host format, set "". -At this time, write CNAME to CDN in public_endpoint. * `streaming_enabled`: Enable streaming uploads, when enabled the file will be sent to the server in chunks as it's being read. This may be unsupported by some providers, try disabling this if you have upload problems. #### Ex AWS S3 settings @@ -604,12 +629,18 @@ This filter replaces the filename (not the path) of an upload. For complete obfu No specific configuration. -#### Pleroma.Upload.Filter.Exiftool +#### Pleroma.Upload.Filter.Exiftool.StripLocation This filter only strips the GPS and location metadata with Exiftool leaving color profiles and attributes intact. No specific configuration. +#### Pleroma.Upload.Filter.Exiftool.ReadDescription + +This filter reads the ImageDescription and iptc:Caption-Abstract fields with Exiftool so clients can prefill the media description field. + +No specific configuration. + #### Pleroma.Upload.Filter.Mogrify * `args`: List of actions for the `mogrify` command like `"strip"` or `["strip", "auto-orient", {"implode", "1"}]`. @@ -747,7 +778,7 @@ Web Push Notifications configuration. You can use the mix task `mix web_push.gen * ``private_key``: VAPID private key ## :logger -* `backends`: `:console` is used to send logs to stdout, `{ExSyslogger, :ex_syslogger}` to log to syslog, and `Quack.Logger` to log to Slack +* `backends`: `:console` is used to send logs to stdout, `{ExSyslogger, :ex_syslogger}` to log to syslog An example to enable ONLY ExSyslogger (f/ex in ``prod.secret.exs``) with info and debug suppressed: ```elixir @@ -770,10 +801,10 @@ config :logger, :ex_syslogger, See: [logger’s documentation](https://hexdocs.pm/logger/Logger.html) and [ex_syslogger’s documentation](https://hexdocs.pm/ex_syslogger/) -An example of logging info to local syslog, but warn to a Slack channel: +An example of logging info to local syslog, but debug to console: ```elixir config :logger, - backends: [ {ExSyslogger, :ex_syslogger}, Quack.Logger ], + backends: [ {ExSyslogger, :ex_syslogger}, :console ], level: :info config :logger, :ex_syslogger, @@ -781,14 +812,12 @@ config :logger, :ex_syslogger, ident: "pleroma", format: "$metadata[$level] $message" -config :quack, - level: :warn, - meta: [:all], - webhook_url: "https://hooks.slack.com/services/YOUR-API-KEY-HERE" +config :logger, :console, + level: :debug, + format: "\n$time $metadata[$level] $message\n", + metadata: [:request_id] ``` -See the [Quack Github](https://github.com/azohra/quack) for more details - ## Database options @@ -850,13 +879,13 @@ config :pleroma, :admin_token, "somerandomtoken" You can then do ```shell -curl "http://localhost:4000/api/pleroma/admin/users/invites?admin_token=somerandomtoken" +curl "http://localhost:4000/api/v1/pleroma/admin/users/invites?admin_token=somerandomtoken" ``` or ```shell -curl -H "X-Admin-Token: somerandomtoken" "http://localhost:4000/api/pleroma/admin/users/invites" +curl -H "X-Admin-Token: somerandomtoken" "http://localhost:4000/api/v1/pleroma/admin/users/invites" ``` Warning: it's discouraged to use this feature because of the associated security risk: static / rarely changed instance-wide token is much weaker compared to email-password pair of a real admin user; consider using HTTP Basic Auth or OAuth-based authentication instead. @@ -895,6 +924,22 @@ Pleroma account will be created with the same name as the LDAP user name. Note, if your LDAP server is an Active Directory server the correct value is commonly `uid: "cn"`, but if you use an OpenLDAP server the value may be `uid: "uid"`. +### :oauth2 (Pleroma as OAuth 2.0 provider settings) + +OAuth 2.0 provider settings: + +* `token_expires_in` - The lifetime in seconds of the access token. +* `issue_new_refresh_token` - Keeps old refresh token or generate new refresh token when to obtain an access token. +* `clean_expired_tokens` - Enable a background job to clean expired oauth tokens. Defaults to `false`. + +OAuth 2.0 provider and related endpoints: + +* `POST /api/v1/apps` creates client app basing on provided params. +* `GET/POST /oauth/authorize` renders/submits authorization form. +* `POST /oauth/token` creates/renews OAuth token. +* `POST /oauth/revoke` revokes provided OAuth token. +* `GET /api/v1/accounts/verify_credentials` (with proper `Authorization` header or `access_token` URI param) returns user info on requester (with `acct` field containing local nickname and `fqn` field containing fully-qualified nickname which could generally be used as email stub for OAuth software that demands email field in identity endpoint response, like Peertube). + ### OAuth consumer mode OAuth consumer mode allows sign in / sign up via external OAuth providers (e.g. Twitter, Facebook, Google, Microsoft, etc.). @@ -967,14 +1012,6 @@ config :ueberauth, Ueberauth, ] ``` -### OAuth 2.0 provider - :oauth2 - -Configure OAuth 2 provider capabilities: - -* `token_expires_in` - The lifetime in seconds of the access token. -* `issue_new_refresh_token` - Keeps old refresh token or generate new refresh token when to obtain an access token. -* `clean_expired_tokens` - Enable a background job to clean expired oauth tokens. Defaults to `false`. - ## Link parsing ### :uri_schemes @@ -1113,3 +1150,15 @@ Settings to enable and configure expiration for ephemeral activities * `:enabled` - enables ephemeral activities creation * `:min_lifetime` - minimum lifetime for ephemeral activities (in seconds). Default: 10 minutes. + +## ConcurrentLimiter + +Settings to restrict concurrently running jobs. Jobs which can be configured: + +* `Pleroma.Web.RichMedia.Helpers` - generating link previews of URLs in activities +* `Pleroma.Web.ActivityPub.MRF.MediaProxyWarmingPolicy` - warming remote media cache via MediaProxyWarmingPolicy + +Each job has these settings: + +* `:max_running` - max concurrently runnings jobs +* `:max_waiting` - max waiting jobs diff --git a/docs/configuration/how_to_serve_another_domain_for_webfinger.md b/docs/configuration/how_to_serve_another_domain_for_webfinger.md new file mode 100644 index 000000000..5ae3e7943 --- /dev/null +++ b/docs/configuration/how_to_serve_another_domain_for_webfinger.md @@ -0,0 +1,62 @@ +# How to use a different domain name for Pleroma and the users it serves + +Pleroma users are primarily identified by a `user@example.org` handle, and you might want this identifier to be the same as your email or jabber account, for instance. +However, in this case, you are almost certainly serving some web content on `https://example.org` already, and you might want to use another domain (say `pleroma.example.org`) for Pleroma itself. + +Pleroma supports that, but it might be tricky to set up, and any error might prevent you from federating with other instances. + +*If you are already running Pleroma on `example.org`, it is no longer possible to move it to `pleroma.example.org`.* + +## Account identifiers + +It is important to understand that for federation purposes, a user in Pleroma has two unique identifiers associated: + +- A webfinger `acct:` URI, used for discovery and as a verifiable global name for the user across Pleroma instances. In our example, our account's acct: URI is `acct:user@example.org` +- An author/actor URI, used in every other aspect of federation. This is the way in which users are identified in ActivityPub, the underlying protocol used for federation with other Pleroma instances. +In our case, it is `https://pleroma.example.org/users/user`. + +Both account identifiers are unique and required for Pleroma. An important risk if you set up your Pleroma instance incorrectly is to create two users (with different acct: URIs) with conflicting author/actor URIs. + +## WebFinger + +As said earlier, each Pleroma user has an `acct`: URI, which is used for discovery and authentication. When you add @user@example.org, a webfinger query is performed. This is done in two steps: + +1. Querying `https://example.org/.well-known/host-meta` (where the domain of the URL matches the domain part of the `acct`: URI) to get information on how to perform the query. +This file will indeed contain a URL template of the form `https://example.org/.well-known/webfinger?resource={uri}` that will be used in the second step. +2. Fill the returned template with the `acct`: URI to be queried and perform the query: `https://example.org/.well-known/webfinger?resource=acct:user@example.org` + +## Configuring your Pleroma instance + +**_DO NOT ATTEMPT TO CONFIGURE YOUR INSTANCE THIS WAY IF YOU DID NOT UNDERSTAND THE ABOVE_** + +### Configuring Pleroma + +Pleroma has a two configuration settings to enable using different domains for your users and Pleroma itself. `host` in `Pleroma.Web.Endpoint` and `domain` in `Pleroma.Web.WebFinger`. When the latter is not set, it defaults to the value of `host`. + +*Be extra careful when configuring your Pleroma instance, as changing `host` may cause remote instances to register different accounts with the same author/actor URI, which will result in federation issues!* + +```elixir +config :pleroma, Pleroma.Web.Endpoint, + url: [host: "pleroma.example.org"] + +config :pleroma, Pleroma.Web.WebFinger, domain: "example.org" +``` + +- `domain` - is the domain for which your Pleroma instance has authority, it's the domain used in `acct:` URI. In our example, `domain` would be set to `example.org`. This is used in WebFinger account ids, which are the canonical account identifier in some other fediverse software like Mastodon. **If you change `domain`, the accounts on your server will be shown as different accounts in those software**. +- `host` - is the domain used for any URL generated for your instance, including the author/actor URL's. In our case, that would be `pleroma.example.org`. This is used in AP ids, which are the canonical account identifier in Pleroma and some other fediverse software. **You should not change this after you have set up the instance**. + +### Configuring WebFinger domain + +Now, you have Pleroma running at `https://pleroma.example.org` as well as a website at `https://example.org`. If you recall how webfinger queries work, the first step is to query `https://example.org/.well-known/host-meta`, which will contain an URL template. + +Therefore, the easiest way to configure `example.org` is to redirect `/.well-known/host-meta` to `pleroma.example.org`. + +With nginx, it would be as simple as adding: + +```nginx +location = /.well-known/host-meta { + return 301 https://pleroma.example.org$request_uri; +} +``` + +in example.org's server block. diff --git a/docs/configuration/howto_database_config.md b/docs/configuration/howto_database_config.md index ae1462f9b..e5af9097a 100644 --- a/docs/configuration/howto_database_config.md +++ b/docs/configuration/howto_database_config.md @@ -59,7 +59,7 @@ The configuration of Pleroma has traditionally been managed with a config file, Here is an example of a server config stripped down after migration: ``` - use Mix.Config + import Config config :pleroma, Pleroma.Web.Endpoint, url: [host: "cool.pleroma.site", scheme: "https", port: 443] diff --git a/docs/configuration/howto_search_cjk.md b/docs/configuration/howto_search_cjk.md new file mode 100644 index 000000000..a73b10db3 --- /dev/null +++ b/docs/configuration/howto_search_cjk.md @@ -0,0 +1,42 @@ +# How to enable text search for Chinese, Japanese and Korean + +Pleroma's full text search feature is powered by PostgreSQL's native [text search](https://www.postgresql.org/docs/current/textsearch.html), it works well out of box for most of languages, but needs extra configurations for some asian languages like Chinese, Japanese and Korean (CJK). + + +## Setup and test the new search config + +In most cases, you would need an extension installed to support parsing CJK text. Here are a few extensions you may choose from, or you are more than welcome to share additional ones you found working for you with the rest of Pleroma community. + + * [a generic n-gram parser](https://github.com/huangjimmy/pg_cjk_parser) supports Simplifed/Traditional Chinese, Japanese, and Korean + * [a Korean parser](https://github.com/i0seph/textsearch_ko) based on mecab + * [a Japanese parser](https://www.amris.co.jp/tsja/index.html) based on mecab + * [zhparser](https://github.com/amutu/zhparser/) is a PostgreSQL extension base on the Simple Chinese Word Segmentation(SCWS) + * [another Chinese parser](https://github.com/jaiminpan/pg_jieba) based on Jieba Chinese Word Segmentation + +Once you have the new search config , make sure you test it with the `pleroma` user in PostgreSQL (change `YOUR.CONFIG` to your real configuration name) +``` +SELECT ts_debug('YOUR.CONFIG', '安装和配置Nginx, ElixirとErlangをインストールします'); +``` +Check output of the query, and see if it matches your expectation. + + +## Update text search config and index in database + +=== "OTP" + + ```sh + ./bin/pleroma_ctl database set_text_search_config YOUR.CONFIG + ``` + +=== "From Source" + + ```sh + mix pleroma.database set_text_search_config YOUR.CONFIG + ``` + +Note: index update may take a while, and it can be done while the instance is up and running, so you may restart db connection as soon as you see `Recreate index` in task output. + +## Restart database connection +Since some changes above will only apply with a new database connection, you will have to restart either Pleroma or PostgreSQL process, or use `pg_terminate_backend` SQL command without restarting either. + +Now the search results of statuses should be much more friendly for your language of choice, the results for searching users and tags were not changed, as the default parsing/matching should work for most cases. diff --git a/docs/configuration/mrf.md b/docs/configuration/mrf.md index 31c66e098..a31c26b9c 100644 --- a/docs/configuration/mrf.md +++ b/docs/configuration/mrf.md @@ -55,18 +55,18 @@ Servers should be configured as lists. ### Example -This example will enable `SimplePolicy`, block media from `illegalporn.biz`, mark media as NSFW from `porn.biz` and `porn.business`, reject messages from `spam.com`, remove messages from `spam.university` from the federated timeline and block reports (flags) from `whiny.whiner`: +This example will enable `SimplePolicy`, block media from `illegalporn.biz`, mark media as NSFW from `porn.biz` and `porn.business`, reject messages from `spam.com`, remove messages from `spam.university` from the federated timeline and block reports (flags) from `whiny.whiner`. We also give a reason why the moderation was done: ```elixir config :pleroma, :mrf, policies: [Pleroma.Web.ActivityPub.MRF.SimplePolicy] config :pleroma, :mrf_simple, - media_removal: ["illegalporn.biz"], - media_nsfw: ["porn.biz", "porn.business"], - reject: ["spam.com"], - federated_timeline_removal: ["spam.university"], - report_removal: ["whiny.whiner"] + media_removal: [{"illegalporn.biz", "Media can contain illegal contant"}], + media_nsfw: [{"porn.biz", "unmarked nsfw media"}, {"porn.business", "A lot of unmarked nsfw media"}], + reject: [{"spam.com", "They keep spamming our users"}], + federated_timeline_removal: [{"spam.university", "Annoying low-quality posts who otherwise fill up TWKN"}], + report_removal: [{"whiny.whiner", "Keep spamming us with irrelevant reports"}] ``` ### Use with Care @@ -82,7 +82,7 @@ For example, here is a sample policy module which rewrites all messages to "new ```elixir defmodule Pleroma.Web.ActivityPub.MRF.RewritePolicy do @moduledoc "MRF policy which rewrites all Notes to have 'new message content'." - @behaviour Pleroma.Web.ActivityPub.MRF + @behaviour Pleroma.Web.ActivityPub.MRF.Policy # Catch messages which contain Note objects with actual data to filter. # Capture the object as `object`, the message content as `content` and the @@ -133,3 +133,26 @@ config :pleroma, :mrf, ``` Please note that the Pleroma developers consider custom MRF policy modules to fall under the purview of the AGPL. As such, you are obligated to release the sources to your custom MRF policy modules upon request. + +### MRF policies descriptions + +If MRF policy depends on config, it can be added into MRF tab to adminFE by adding `config_description/0` method, which returns a map with a specific structure. See existing MRF's like `lib/pleroma/web/activity_pub/mrf/activity_expiration_policy.ex` for examples. Note that more complex inputs, like tuples or maps, may need extra changes in the adminFE and just adding it to `config_description/0` may not be enough to get these inputs working from the adminFE. + +Example: + +```elixir +%{ + key: :mrf_activity_expiration, + related_policy: "Pleroma.Web.ActivityPub.MRF.ActivityExpirationPolicy", + label: "MRF Activity Expiration Policy", + description: "Adds automatic expiration to all local activities", + children: [ + %{ + key: :days, + type: :integer, + description: "Default global expiration time for all local activities (in days)", + suggestions: [90, 365] + } + ] + } +``` diff --git a/docs/configuration/postgresql.md b/docs/configuration/postgresql.md index 6983fb459..e251eb83b 100644 --- a/docs/configuration/postgresql.md +++ b/docs/configuration/postgresql.md @@ -1,10 +1,28 @@ -# Optimizing your PostgreSQL performance +# Optimizing PostgreSQL performance -Pleroma performance depends to a large extent on good database performance. The default PostgreSQL settings are mostly fine, but often you can get better performance by changing a few settings. +Pleroma performance is largely dependent on performance of the underlying database. Better performance can be achieved by adjusting a few settings. -You can use [PGTune](https://pgtune.leopard.in.ua) to get recommendations for your setup. If you do, set the "Number of Connections" field to 20, as Pleroma will only use 10 concurrent connections anyway. If you don't, it will give you advice that might even hurt your performance. +## PGTune -We also recommend not using the "Network Storage" option. +[PgTune](https://pgtune.leopard.in.ua) can be used to get recommended settings. Be sure to set "Number of Connections" to 20, otherwise it might produce settings hurtful to database performance. It is also recommended to not use "Network Storage" option. + +## Disable generic query plans + +When PostgreSQL receives a query, it decides on a strategy for searching the requested data, this is called a query plan. The query planner has two modes: generic and custom. Generic makes a plan for all queries of the same shape, ignoring the parameters, which is then cached and reused. Custom, on the contrary, generates a unique query plan based on query parameters. + +By default PostgreSQL has an algorithm to decide which mode is more efficient for particular query, however this algorithm has been observed to be wrong on some of the queries Pleroma sends, leading to serious performance loss. Therefore, it is recommended to disable generic mode. + + +Pleroma already avoids generic query plans by default, however the method it uses is not the most efficient because it needs to be compatible with all supported PostgreSQL versions. For PostgreSQL 12 and higher additional performance can be gained by adding the following to Pleroma configuration: +```elixir +config :pleroma, Pleroma.Repo, + prepare: :named, + parameters: [ + plan_cache_mode: "force_custom_plan" + ] +``` + +A more detailed explaination of the issue can be found at <https://blog.soykaf.com/post/postgresql-elixir-troubles/>. ## Example configurations @@ -28,4 +46,3 @@ max_worker_processes = 2 max_parallel_workers_per_gather = 1 max_parallel_workers = 2 ``` - |