From 43dfa58ebda407a0813d398bee8d0ae3e5c9fd5b Mon Sep 17 00:00:00 2001 From: Claudio Maradonna Date: Mon, 11 Apr 2022 15:10:01 +0200 Subject: added tests for ipfs uploader. adapted changelog.md accordingly. improved ipfs uploader with external suggestions fix lint description.exs --- docs/configuration/cheatsheet.md | 13 +++++++++++++ 1 file changed, 13 insertions(+) (limited to 'docs/configuration') diff --git a/docs/configuration/cheatsheet.md b/docs/configuration/cheatsheet.md index 74642397b..7e1f9c934 100644 --- a/docs/configuration/cheatsheet.md +++ b/docs/configuration/cheatsheet.md @@ -614,6 +614,19 @@ config :ex_aws, :s3, host: "s3.eu-central-1.amazonaws.com" ``` +#### Pleroma.Uploaders.IPFS + +* `post_gateway_url`: URL with port of POST Gateway (unauthenticated) +* `get_gateway_url`: URL of public GET Gateway + +Example: + +```elixir +config :pleroma, Pleroma.Uploaders.IPFS, + post_gateway_url: "http://localhost:5001", + get_gateway_url: "http://<%= cid %>.ipfs.mydomain.com" +``` + ### Upload filters #### Pleroma.Upload.Filter.AnonymizeFilename -- cgit v1.2.3 From 44659ecd65fb2251f9130fcecf1732b8931104c1 Mon Sep 17 00:00:00 2001 From: Claudio Maradonna Date: Sat, 16 Apr 2022 09:38:49 +0200 Subject: ipfs: revert to String.replace for cid placeholder ipfs: fix lint --- docs/configuration/cheatsheet.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'docs/configuration') diff --git a/docs/configuration/cheatsheet.md b/docs/configuration/cheatsheet.md index 7e1f9c934..d35b33574 100644 --- a/docs/configuration/cheatsheet.md +++ b/docs/configuration/cheatsheet.md @@ -624,7 +624,7 @@ Example: ```elixir config :pleroma, Pleroma.Uploaders.IPFS, post_gateway_url: "http://localhost:5001", - get_gateway_url: "http://<%= cid %>.ipfs.mydomain.com" + get_gateway_url: "http://{CID}.ipfs.mydomain.com" ``` ### Upload filters -- cgit v1.2.3 From 07ccab9766a6289326676a4814537564f25f35fa Mon Sep 17 00:00:00 2001 From: Ekaterina Vaartis Date: Mon, 23 Aug 2021 20:27:16 +0300 Subject: Add search/meilisearch documentation --- docs/configuration/search.md | 99 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 99 insertions(+) create mode 100644 docs/configuration/search.md (limited to 'docs/configuration') diff --git a/docs/configuration/search.md b/docs/configuration/search.md new file mode 100644 index 000000000..14ec2bc63 --- /dev/null +++ b/docs/configuration/search.md @@ -0,0 +1,99 @@ +# Configuring search + +{! backend/administration/CLI_tasks/general_cli_task_info.include !} + +## Built-in search + +To use built-in search that has no external dependencies, set the search module to `Pleroma.Activity`: + +> config :pleroma, Pleroma.Search, module: Pleroma.Activity + +While it has no external dependencies, it has problems with performance and relevancy. + +## Meilisearch + +To use [meilisearch](https://www.meilisearch.com/), set the search module to `Pleroma.Search.Meilisearch`: + +> config :pleroma, Pleroma.Search, module: Pleroma.Search.Meilisearch + +You then need to set the address of the meilisearch instance, and optionally the private key for authentication. + +> config :pleroma, Pleroma.Search.Meilisearch, +> url: "http://127.0.0.1:7700/", +> private_key: "private key" + +Information about setting up meilisearch can be found in the +[official documentation](https://docs.meilisearch.com/learn/getting_started/installation.html). +You probably want to start it with `MEILI_NO_ANALYTICS=true` and `MEILI_NO_CENTRY=true` environment variables, +to disable analytics. + +### Private key authentication (optional) + +To set the private key, use the `MEILI_MASTER_KEY` environment variable when starting. After setting the _master key_, +you have to get the _private key_, which is actually used for authentication. + +=== "OTP" + ```sh + ./bin/pleroma_ctl search.meilisearch show-private-key + ``` + +=== "From Source" + ```sh + mix pleroma.search.meilisearch show-private-key + ``` + +This is the key you actually put into your configuration file. + +### Initial indexing + +After setting up the configuration, you'll want to index all of your already existsing posts. Only public posts are indexed. You'll only +have to do it one time, but it might take a while, depending on the amount of posts your instance has seen. This is also a fairly RAM +consuming process for `meilisearch`, and it will take a lot of RAM when running if you have a lot of posts (seems to be around 5G for ~1.2 +million posts while idle and up to 7G while indexing initially, but your experience may be different). + +To start te initial indexing, run the `index` command: + +=== "OTP" + ```sh + ./bin/pleroma_ctl search.meilisearch index + ``` + +=== "From Source" + ```sh + mix pleroma.search.meilisearch index + ``` + +This will show you the total amount of posts to index, and then show you the amount of posts indexed currently, until the numbers eventually +become the same. The posts are indexed in big batches and meilisearch will take some time to actually index them, even after you have +inserted all the posts into it. Depending on the amount of posts, this may be as long as several hours. To get information about the status +of indexing and how many posts have actually been indexed, use the `stats` command: + +=== "OTP" + ```sh + ./bin/pleroma_ctl search.meilisearch stats + ``` + +=== "From Source" + ```sh + mix pleroma.search.meilisearch stats + ``` + +### Clearing the index + +In case you need to clear the index (for example, to re-index from scratch, if that needs to happen for some reason), you can +use the `clear` command: + +=== "OTP" + ```sh + ./bin/pleroma_ctl search.meilisearch clear + ``` + +=== "From Source" + ```sh + mix pleroma.search.meilisearch clear + ``` + +This will clear **all** the posts from the search index. Note, that deleted posts are also removed from index by the instance itself, so +there is no need to actually clear the whole index, unless you want **all** of it gone. That said, the index does not hold any information +that cannot be re-created from the database, it should also generally be a lot smaller than the size of your database. Still, the size +depends on the amount of text in posts. -- cgit v1.2.3 From c569ad05b3d812c87171e68eac79eec749321033 Mon Sep 17 00:00:00 2001 From: Ekaterina Vaartis Date: Tue, 12 Oct 2021 19:14:39 +0300 Subject: Add more documentation about rum to meilisearch docs --- docs/configuration/search.md | 9 +++++++++ 1 file changed, 9 insertions(+) (limited to 'docs/configuration') diff --git a/docs/configuration/search.md b/docs/configuration/search.md index 14ec2bc63..e9743f1a4 100644 --- a/docs/configuration/search.md +++ b/docs/configuration/search.md @@ -12,6 +12,15 @@ While it has no external dependencies, it has problems with performance and rele ## Meilisearch +Note that it's quite a bit more memory hungry than PostgreSQL (around 4-5G for ~1.2 million +posts while idle and up to 7G while indexing initially). The disk usage for this additional index is also +around 4 gigabytes. Like [RUM](./cheatsheet.md#rum-indexing-for-full-text-search) indexes, it offers considerably +higher performance and ordering by timestamp in a reasonable amount of time. +Additionally, the search results seem to be more accurate. + +Due to high memory usage, it may be best to set it up on a different machine, if running pleroma on a low-resource +computer, and use private key authentication to secure the remote search instance. + To use [meilisearch](https://www.meilisearch.com/), set the search module to `Pleroma.Search.Meilisearch`: > config :pleroma, Pleroma.Search, module: Pleroma.Search.Meilisearch -- cgit v1.2.3 From 8898b5e927bae27a521e4eadd0faf970ad27c5bc Mon Sep 17 00:00:00 2001 From: Ekaterina Vaartis Date: Sun, 14 Nov 2021 20:15:12 +0300 Subject: Fix a typo in search docs --- docs/configuration/search.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'docs/configuration') diff --git a/docs/configuration/search.md b/docs/configuration/search.md index e9743f1a4..9adc7884f 100644 --- a/docs/configuration/search.md +++ b/docs/configuration/search.md @@ -60,7 +60,7 @@ have to do it one time, but it might take a while, depending on the amount of po consuming process for `meilisearch`, and it will take a lot of RAM when running if you have a lot of posts (seems to be around 5G for ~1.2 million posts while idle and up to 7G while indexing initially, but your experience may be different). -To start te initial indexing, run the `index` command: +To start the initial indexing, run the `index` command: === "OTP" ```sh -- cgit v1.2.3 From a6946048fbe049aa223d094d36eb767739ab5ff2 Mon Sep 17 00:00:00 2001 From: Ekaterina Vaartis Date: Wed, 17 Nov 2021 22:29:49 +0300 Subject: Rename Activity.Search to Search.DatabaseSearch --- docs/configuration/search.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'docs/configuration') diff --git a/docs/configuration/search.md b/docs/configuration/search.md index 9adc7884f..c7e77d9c2 100644 --- a/docs/configuration/search.md +++ b/docs/configuration/search.md @@ -6,7 +6,7 @@ To use built-in search that has no external dependencies, set the search module to `Pleroma.Activity`: -> config :pleroma, Pleroma.Search, module: Pleroma.Activity +> config :pleroma, Pleroma.Search, module: Pleroma.Search.DatabaseSearch While it has no external dependencies, it has problems with performance and relevancy. -- cgit v1.2.3 From 3412713c5b2fd24605b18933ef70de164ee14f2d Mon Sep 17 00:00:00 2001 From: Ekaterina Vaartis Date: Mon, 20 Dec 2021 18:16:33 +0300 Subject: Update search.md documentation with meilisearch indexing steps --- docs/configuration/search.md | 9 +++++++++ 1 file changed, 9 insertions(+) (limited to 'docs/configuration') diff --git a/docs/configuration/search.md b/docs/configuration/search.md index c7e77d9c2..7dbbd3e17 100644 --- a/docs/configuration/search.md +++ b/docs/configuration/search.md @@ -60,6 +60,15 @@ have to do it one time, but it might take a while, depending on the amount of po consuming process for `meilisearch`, and it will take a lot of RAM when running if you have a lot of posts (seems to be around 5G for ~1.2 million posts while idle and up to 7G while indexing initially, but your experience may be different). +The sequence of actions is as follows: + +1. First, change the configuration to use `Pleroma.Search.Meilisearch` as the search backend +2. Restart your instance, at this point it can be used while the search indexing is running, though search won't return anything +3. Start the initial indexing process (as described below with `index`), + and wait until the task says it sent everything from the database to index +4. Wait until everything is actually indexed (by checking with `stats` as described below), + at this point you don't have to do anything, just wait a while. + To start the initial indexing, run the `index` command: === "OTP" -- cgit v1.2.3 From 4f2637acc6c46ea39ae38e869903e7ffcc38b34d Mon Sep 17 00:00:00 2001 From: Ekaterina Vaartis Date: Mon, 20 Dec 2021 19:27:22 +0300 Subject: Add description for initial_indexing_chunk_size --- docs/configuration/search.md | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) (limited to 'docs/configuration') diff --git a/docs/configuration/search.md b/docs/configuration/search.md index 7dbbd3e17..a785a18ad 100644 --- a/docs/configuration/search.md +++ b/docs/configuration/search.md @@ -25,11 +25,15 @@ To use [meilisearch](https://www.meilisearch.com/), set the search module to `Pl > config :pleroma, Pleroma.Search, module: Pleroma.Search.Meilisearch -You then need to set the address of the meilisearch instance, and optionally the private key for authentication. +You then need to set the address of the meilisearch instance, and optionally the private key for authentication. You might +also want to change the `initial_indexing_chunk_size` to be smaller if you're server is not very powerful, but not higher than `100_000`, +because meilisearch will refuse to process it if it's too big. However, in general you want this to be as big as possible, because meilisearch +indexes faster when it can process many posts in a single batch. > config :pleroma, Pleroma.Search.Meilisearch, > url: "http://127.0.0.1:7700/", -> private_key: "private key" +> private_key: "private key", +> initial_indexing_chunk_size: 100_000 Information about setting up meilisearch can be found in the [official documentation](https://docs.meilisearch.com/learn/getting_started/installation.html). -- cgit v1.2.3 From 1e23f527e3e22108b402552a0766e488048ed3f4 Mon Sep 17 00:00:00 2001 From: Ekaterina Vaartis Date: Tue, 22 Mar 2022 20:29:17 +0300 Subject: Change the meilisearch key auth to conform to 0.25.0 --- docs/configuration/search.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) (limited to 'docs/configuration') diff --git a/docs/configuration/search.md b/docs/configuration/search.md index a785a18ad..82217e5ee 100644 --- a/docs/configuration/search.md +++ b/docs/configuration/search.md @@ -47,15 +47,15 @@ you have to get the _private key_, which is actually used for authentication. === "OTP" ```sh - ./bin/pleroma_ctl search.meilisearch show-private-key + ./bin/pleroma_ctl search.meilisearch show-keys ``` === "From Source" ```sh - mix pleroma.search.meilisearch show-private-key + mix pleroma.search.meilisearch show-keys ``` -This is the key you actually put into your configuration file. +You will see a "Default Admin API Key", this is the key you actually put into your configuration file. ### Initial indexing -- cgit v1.2.3 From b150e6f15e0f06c8e23c0ac66aeaf80eb2f8c31a Mon Sep 17 00:00:00 2001 From: Ekaterina Vaartis Date: Wed, 23 Mar 2022 11:36:01 +0300 Subject: Update meilisearch docs --- docs/configuration/search.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) (limited to 'docs/configuration') diff --git a/docs/configuration/search.md b/docs/configuration/search.md index 82217e5ee..f131948a7 100644 --- a/docs/configuration/search.md +++ b/docs/configuration/search.md @@ -37,8 +37,10 @@ indexes faster when it can process many posts in a single batch. Information about setting up meilisearch can be found in the [official documentation](https://docs.meilisearch.com/learn/getting_started/installation.html). -You probably want to start it with `MEILI_NO_ANALYTICS=true` and `MEILI_NO_CENTRY=true` environment variables, -to disable analytics. +You probably want to start it with `MEILI_NO_ANALYTICS=true` environment variable to disable analytics. +At least version 0.25.0 is required, but you are strongly adviced to use at least 0.26.0, as it introduces +the `--enable-auto-batching` option which drastically improves performance. Without this option, the search +is hardly usable on a somewhat big instance. ### Private key authentication (optional) -- cgit v1.2.3 From 0231a09310895bd2af24eff221de97eb8c92307e Mon Sep 17 00:00:00 2001 From: duponin Date: Sun, 16 Apr 2023 15:50:32 +0200 Subject: Remove SSH/BBS feature from core And link to sshocial, the replacement client for this removed feature --- docs/configuration/cheatsheet.md | 17 ++--------------- 1 file changed, 2 insertions(+), 15 deletions(-) (limited to 'docs/configuration') diff --git a/docs/configuration/cheatsheet.md b/docs/configuration/cheatsheet.md index bbdf30a0f..70abe3e0c 100644 --- a/docs/configuration/cheatsheet.md +++ b/docs/configuration/cheatsheet.md @@ -873,21 +873,8 @@ This will probably take a long time. ### BBS / SSH access -To enable simple command line interface accessible over ssh, add a setting like this to your configuration file: - -```exs -app_dir = File.cwd! -priv_dir = Path.join([app_dir, "priv/ssh_keys"]) - -config :esshd, - enabled: true, - priv_dir: priv_dir, - handler: "Pleroma.BBS.Handler", - port: 10_022, - password_authenticator: "Pleroma.BBS.Authenticator" -``` - -Feel free to adjust the priv_dir and port number. Then you will have to create the key for the keys (in the example `priv/ssh_keys`) and create the host keys with `ssh-keygen -m PEM -N "" -b 2048 -t rsa -f ssh_host_rsa_key`. After restarting, you should be able to connect to your Pleroma instance with `ssh username@server -p $PORT` +This feature has been removed from Pleroma core. +However, a client has been made and is available at https://git.pleroma.social/Duponin/sshocial. ### :gopher * `enabled`: Enables the gopher interface -- cgit v1.2.3 From f970091c6a58d06a42594e2c4a0baa5a86617652 Mon Sep 17 00:00:00 2001 From: tusooa Date: Fri, 26 May 2023 17:17:13 -0400 Subject: Add instructions to serve media on another domain --- docs/configuration/hardening.md | 14 ++++++++++++++ 1 file changed, 14 insertions(+) (limited to 'docs/configuration') diff --git a/docs/configuration/hardening.md b/docs/configuration/hardening.md index d3bfc4e4a..4f40873e6 100644 --- a/docs/configuration/hardening.md +++ b/docs/configuration/hardening.md @@ -62,6 +62,20 @@ An additional “Expect-CT” header will be sent with the configured `ct_max_ag If you click on a link, your browser’s request to the other site will include from where it is coming from. The “Referrer policy” header tells the browser how and if it should send this information. (see [Referrer policy](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Referrer-Policy)) +### Uploaded media and media proxy + +It is STRONGLY RECOMMENDED to serve both the locally-uploaded media and the media proxy from another domain than the domain that Pleroma runs on, if applicable. + +```elixir +config :pleroma, :media_proxy, + base_url: "https://some.other.domain" + +config :pleroma, Pleroma.Upload, + base_url: "https://some.other.domain" +``` + +See `installation/pleroma-mediaproxy.nginx` for examples on how to configure your media proxy. + ## systemd A systemd unit example is provided at `installation/pleroma.service`. -- cgit v1.2.3 From 9caa0b0be1feffb20b3eaccb48298de572d32393 Mon Sep 17 00:00:00 2001 From: Mark Felder Date: Mon, 29 May 2023 15:46:49 -0400 Subject: Add OnlyMedia Upload Filter to simplify restricting uploads to audio, image, and video types --- docs/configuration/cheatsheet.md | 6 ++++++ 1 file changed, 6 insertions(+) (limited to 'docs/configuration') diff --git a/docs/configuration/cheatsheet.md b/docs/configuration/cheatsheet.md index 70abe3e0c..026753f38 100644 --- a/docs/configuration/cheatsheet.md +++ b/docs/configuration/cheatsheet.md @@ -671,6 +671,12 @@ This filter reads the ImageDescription and iptc:Caption-Abstract fields with Exi No specific configuration. +#### Pleroma.Upload.Filter.OnlyMedia + +This filter rejects uploads that are not identified with Content-Type matching audio/*, image/*, or video/* + +No specific configuration. + #### Pleroma.Upload.Filter.Mogrify * `args`: List of actions for the `mogrify` command like `"strip"` or `["strip", "auto-orient", {"implode", "1"}]`. -- cgit v1.2.3 From 50a20f3bbd1203679c9295eb6002225dd630baf8 Mon Sep 17 00:00:00 2001 From: Mark Felder Date: Mon, 29 May 2023 15:53:16 -0400 Subject: Esacpe the asterisks in Markdown --- docs/configuration/cheatsheet.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'docs/configuration') diff --git a/docs/configuration/cheatsheet.md b/docs/configuration/cheatsheet.md index 026753f38..1e49a79d0 100644 --- a/docs/configuration/cheatsheet.md +++ b/docs/configuration/cheatsheet.md @@ -673,7 +673,7 @@ No specific configuration. #### Pleroma.Upload.Filter.OnlyMedia -This filter rejects uploads that are not identified with Content-Type matching audio/*, image/*, or video/* +This filter rejects uploads that are not identified with Content-Type matching audio/\*, image/\*, or video/\* No specific configuration. -- cgit v1.2.3 From a2bbd7c9dadad42c995e7627a559673f04c2618b Mon Sep 17 00:00:00 2001 From: Sean King Date: Wed, 31 May 2023 12:22:13 -0600 Subject: Fix base media and proxy URL in instructions to serve media on another domain --- docs/configuration/hardening.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) (limited to 'docs/configuration') diff --git a/docs/configuration/hardening.md b/docs/configuration/hardening.md index 4f40873e6..493ba608c 100644 --- a/docs/configuration/hardening.md +++ b/docs/configuration/hardening.md @@ -68,10 +68,10 @@ It is STRONGLY RECOMMENDED to serve both the locally-uploaded media and the medi ```elixir config :pleroma, :media_proxy, - base_url: "https://some.other.domain" + base_url: "https://some.other.domain/proxy" config :pleroma, Pleroma.Upload, - base_url: "https://some.other.domain" + base_url: "https://some.other.domain/media" ``` See `installation/pleroma-mediaproxy.nginx` for examples on how to configure your media proxy. -- cgit v1.2.3 From c9cb90ff4fac5bf645720c7e7ef0bf4b6acf8afa Mon Sep 17 00:00:00 2001 From: Sean King Date: Wed, 31 May 2023 17:49:06 -0600 Subject: Media proxy base URL doesn't need /proxy --- docs/configuration/hardening.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'docs/configuration') diff --git a/docs/configuration/hardening.md b/docs/configuration/hardening.md index 493ba608c..cc46d1ff9 100644 --- a/docs/configuration/hardening.md +++ b/docs/configuration/hardening.md @@ -68,7 +68,7 @@ It is STRONGLY RECOMMENDED to serve both the locally-uploaded media and the medi ```elixir config :pleroma, :media_proxy, - base_url: "https://some.other.domain/proxy" + base_url: "https://some.other.domain" config :pleroma, Pleroma.Upload, base_url: "https://some.other.domain/media" -- cgit v1.2.3 From 18a8378beb01a7ee93afcddf16e2699c5fe6d35a Mon Sep 17 00:00:00 2001 From: tusooa Date: Tue, 28 Feb 2023 12:05:54 -0500 Subject: Update config cheatsheet --- docs/configuration/cheatsheet.md | 5 +++++ 1 file changed, 5 insertions(+) (limited to 'docs/configuration') diff --git a/docs/configuration/cheatsheet.md b/docs/configuration/cheatsheet.md index 1e49a79d0..696d10e74 100644 --- a/docs/configuration/cheatsheet.md +++ b/docs/configuration/cheatsheet.md @@ -261,6 +261,11 @@ Notes: * `follower_nickname`: The name of the bot account to use for following newly discovered users. Using `followbot` or similar is strongly suggested. +#### :mrf_emoji +* `remove_url`: A list of patterns which result in emoji whose URL matches being removed from the message. This will apply to both statuses and user profiles. Each pattern can be a string or a [regular expression](https://hexdocs.pm/elixir/Regex.html). +* `remove_shortcode`: A list of patterns which result in emoji whose shortcode matches being removed from the message. This will apply to both statuses and user profiles. Each pattern can be a string or a [regular expression](https://hexdocs.pm/elixir/Regex.html). +* `federated_timeline_removal_url`: A list of patterns which result in message with emojis whose URLs match being removed from federated timelines (a.k.a unlisted). This will apply only to statuses. Each pattern can be a string or a [regular expression](https://hexdocs.pm/elixir/Regex.html). +* `federated_timeline_removal_shortcode`: A list of patterns which result in message with emojis whose shortcodes match being removed from federated timelines (a.k.a unlisted). This will apply only to statuses. Each pattern can be a string or a [regular expression](https://hexdocs.pm/elixir/Regex.html). ### :activitypub * `unfollow_blocked`: Whether blocks result in people getting unfollowed -- cgit v1.2.3 From ef8a6c539a98633d1a78fa42379952b93f847309 Mon Sep 17 00:00:00 2001 From: tusooa Date: Tue, 13 Jun 2023 14:53:20 -0400 Subject: Make EmojiPolicy aware of custom emoji reactions --- docs/configuration/cheatsheet.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) (limited to 'docs/configuration') diff --git a/docs/configuration/cheatsheet.md b/docs/configuration/cheatsheet.md index 696d10e74..f43cde114 100644 --- a/docs/configuration/cheatsheet.md +++ b/docs/configuration/cheatsheet.md @@ -262,8 +262,8 @@ Notes: * `follower_nickname`: The name of the bot account to use for following newly discovered users. Using `followbot` or similar is strongly suggested. #### :mrf_emoji -* `remove_url`: A list of patterns which result in emoji whose URL matches being removed from the message. This will apply to both statuses and user profiles. Each pattern can be a string or a [regular expression](https://hexdocs.pm/elixir/Regex.html). -* `remove_shortcode`: A list of patterns which result in emoji whose shortcode matches being removed from the message. This will apply to both statuses and user profiles. Each pattern can be a string or a [regular expression](https://hexdocs.pm/elixir/Regex.html). +* `remove_url`: A list of patterns which result in emoji whose URL matches being removed from the message. This will apply to statuses, emoji reactions, and user profiles. Each pattern can be a string or a [regular expression](https://hexdocs.pm/elixir/Regex.html). +* `remove_shortcode`: A list of patterns which result in emoji whose shortcode matches being removed from the message. This will apply to statuses, emoji reactions, and user profiles. Each pattern can be a string or a [regular expression](https://hexdocs.pm/elixir/Regex.html). * `federated_timeline_removal_url`: A list of patterns which result in message with emojis whose URLs match being removed from federated timelines (a.k.a unlisted). This will apply only to statuses. Each pattern can be a string or a [regular expression](https://hexdocs.pm/elixir/Regex.html). * `federated_timeline_removal_shortcode`: A list of patterns which result in message with emojis whose shortcodes match being removed from federated timelines (a.k.a unlisted). This will apply only to statuses. Each pattern can be a string or a [regular expression](https://hexdocs.pm/elixir/Regex.html). -- cgit v1.2.3 From 57ef1d121101d785c043ef6aaf2d33bb9be3ec3b Mon Sep 17 00:00:00 2001 From: Alex Gleason Date: Mon, 24 Jan 2022 16:44:35 -0600 Subject: Add InlineQuotePolicy to force quote URLs inline --- docs/configuration/cheatsheet.md | 4 ++++ 1 file changed, 4 insertions(+) (limited to 'docs/configuration') diff --git a/docs/configuration/cheatsheet.md b/docs/configuration/cheatsheet.md index f43cde114..32cc5811a 100644 --- a/docs/configuration/cheatsheet.md +++ b/docs/configuration/cheatsheet.md @@ -160,6 +160,7 @@ To add configuration to your config file, you can copy it from the base config. * `Pleroma.Web.ActivityPub.MRF.AntiFollowbotPolicy`: Drops follow requests from followbots. Users can still allow bots to follow them by first following the bot. * `Pleroma.Web.ActivityPub.MRF.KeywordPolicy`: Rejects or removes from the federated timeline or replaces keywords. (See [`:mrf_keyword`](#mrf_keyword)). * `Pleroma.Web.ActivityPub.MRF.ForceMentionsInContent`: Forces every mentioned user to be reflected in the post content. + * `Pleroma.Web.ActivityPub.MRF.InlineQuotePolicy`: Forces quote post URLs to be reflected in the message content inline. * `transparency`: Make the content of your Message Rewrite Facility settings public (via nodeinfo). * `transparency_exclusions`: Exclude specific instance names from MRF transparency. The use of the exclusions feature will be disclosed in nodeinfo as a boolean value. @@ -267,6 +268,9 @@ Notes: * `federated_timeline_removal_url`: A list of patterns which result in message with emojis whose URLs match being removed from federated timelines (a.k.a unlisted). This will apply only to statuses. Each pattern can be a string or a [regular expression](https://hexdocs.pm/elixir/Regex.html). * `federated_timeline_removal_shortcode`: A list of patterns which result in message with emojis whose shortcodes match being removed from federated timelines (a.k.a unlisted). This will apply only to statuses. Each pattern can be a string or a [regular expression](https://hexdocs.pm/elixir/Regex.html). +#### :mrf_inline_quote +* `prefix`: Prefix before the link (default: `RT`) + ### :activitypub * `unfollow_blocked`: Whether blocks result in people getting unfollowed * `outgoing_blocks`: Whether to federate blocks to other instances -- cgit v1.2.3 From 163e5637335f9454688d3cc83530f82fc640a5b9 Mon Sep 17 00:00:00 2001 From: tusooa Date: Wed, 12 Jul 2023 09:30:43 -0400 Subject: Allow more flexibility in InlineQuotePolicy --- docs/configuration/cheatsheet.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'docs/configuration') diff --git a/docs/configuration/cheatsheet.md b/docs/configuration/cheatsheet.md index 32cc5811a..a17f8735a 100644 --- a/docs/configuration/cheatsheet.md +++ b/docs/configuration/cheatsheet.md @@ -269,7 +269,7 @@ Notes: * `federated_timeline_removal_shortcode`: A list of patterns which result in message with emojis whose shortcodes match being removed from federated timelines (a.k.a unlisted). This will apply only to statuses. Each pattern can be a string or a [regular expression](https://hexdocs.pm/elixir/Regex.html). #### :mrf_inline_quote -* `prefix`: Prefix before the link (default: `RT`) +* `template`: The template to append to the post. `{url}` will be replaced with the actual link to the quoted post. Default: `RT: {url}` ### :activitypub * `unfollow_blocked`: Whether blocks result in people getting unfollowed -- cgit v1.2.3 From e349e92a441840bbbdbf13cacd307e65f85a38ff Mon Sep 17 00:00:00 2001 From: tusooa Date: Wed, 12 Jul 2023 14:27:29 -0400 Subject: Add mrf to force link tag of quoting posts --- docs/configuration/cheatsheet.md | 1 + 1 file changed, 1 insertion(+) (limited to 'docs/configuration') diff --git a/docs/configuration/cheatsheet.md b/docs/configuration/cheatsheet.md index a17f8735a..a4cae4dbb 100644 --- a/docs/configuration/cheatsheet.md +++ b/docs/configuration/cheatsheet.md @@ -161,6 +161,7 @@ To add configuration to your config file, you can copy it from the base config. * `Pleroma.Web.ActivityPub.MRF.KeywordPolicy`: Rejects or removes from the federated timeline or replaces keywords. (See [`:mrf_keyword`](#mrf_keyword)). * `Pleroma.Web.ActivityPub.MRF.ForceMentionsInContent`: Forces every mentioned user to be reflected in the post content. * `Pleroma.Web.ActivityPub.MRF.InlineQuotePolicy`: Forces quote post URLs to be reflected in the message content inline. + * `Pleroma.Web.ActivityPub.MRF.QuoteToLinkTagPolicy`: Force a Link tag for posts quoting another post. (may break outgoing federation of quote posts with older Pleroma versions) * `transparency`: Make the content of your Message Rewrite Facility settings public (via nodeinfo). * `transparency_exclusions`: Exclude specific instance names from MRF transparency. The use of the exclusions feature will be disclosed in nodeinfo as a boolean value. -- cgit v1.2.3 From 086ba59d0346be870dc7df2660fbb55666bf0af7 Mon Sep 17 00:00:00 2001 From: "Haelwenn (lanodan) Monnier" Date: Sat, 16 Dec 2023 18:56:46 +0100 Subject: HTTPSignaturePlug: Add :authorized_fetch_mode_exceptions --- docs/configuration/cheatsheet.md | 1 + 1 file changed, 1 insertion(+) (limited to 'docs/configuration') diff --git a/docs/configuration/cheatsheet.md b/docs/configuration/cheatsheet.md index a4cae4dbb..06933ba76 100644 --- a/docs/configuration/cheatsheet.md +++ b/docs/configuration/cheatsheet.md @@ -279,6 +279,7 @@ Notes: * `deny_follow_blocked`: Whether to disallow following an account that has blocked the user in question * `sign_object_fetches`: Sign object fetches with HTTP signatures * `authorized_fetch_mode`: Require HTTP signatures for AP fetches +* `authorized_fetch_mode_exceptions`: List of IPs (CIDR format accepted) to exempt from HTTP Signatures requirement (for example to allow debugging, you shouldn't otherwise need this) ## Pleroma.User -- cgit v1.2.3 From 017e35fbf128d47c033275a70b76b72f24d7c754 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?marcin=20miko=C5=82ajczak?= Date: Thu, 28 Dec 2023 00:15:32 +0100 Subject: Fix some more typos MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: marcin mikołajczak --- docs/configuration/cheatsheet.md | 8 ++++---- docs/configuration/custom_emoji.md | 2 +- docs/configuration/i2p.md | 2 +- docs/configuration/onion_federation.md | 2 +- docs/configuration/optimizing_beam.md | 2 +- docs/configuration/postgresql.md | 2 +- docs/configuration/search.md | 4 ++-- 7 files changed, 11 insertions(+), 11 deletions(-) (limited to 'docs/configuration') diff --git a/docs/configuration/cheatsheet.md b/docs/configuration/cheatsheet.md index a4cae4dbb..7bba7b26e 100644 --- a/docs/configuration/cheatsheet.md +++ b/docs/configuration/cheatsheet.md @@ -154,7 +154,7 @@ To add configuration to your config file, you can copy it from the base config. * `Pleroma.Web.ActivityPub.MRF.MentionPolicy`: Drops posts mentioning configurable users. (See [`:mrf_mention`](#mrf_mention)). * `Pleroma.Web.ActivityPub.MRF.VocabularyPolicy`: Restricts activities to a configured set of vocabulary. (See [`:mrf_vocabulary`](#mrf_vocabulary)). * `Pleroma.Web.ActivityPub.MRF.ObjectAgePolicy`: Rejects or delists posts based on their age when received. (See [`:mrf_object_age`](#mrf_object_age)). - * `Pleroma.Web.ActivityPub.MRF.ActivityExpirationPolicy`: Sets a default expiration on all posts made by users of the local instance. Requires `Pleroma.Workers.PurgeExpiredActivity` to be enabled for processing the scheduled delections. + * `Pleroma.Web.ActivityPub.MRF.ActivityExpirationPolicy`: Sets a default expiration on all posts made by users of the local instance. Requires `Pleroma.Workers.PurgeExpiredActivity` to be enabled for processing the scheduled deletions. * `Pleroma.Web.ActivityPub.MRF.ForceBotUnlistedPolicy`: Makes all bot posts to disappear from public timelines. * `Pleroma.Web.ActivityPub.MRF.FollowBotPolicy`: Automatically follows newly discovered users from the specified bot account. Local accounts, locked accounts, and users with "#nobot" in their bio are respected and excluded from being followed. * `Pleroma.Web.ActivityPub.MRF.AntiFollowbotPolicy`: Drops follow requests from followbots. Users can still allow bots to follow them by first following the bot. @@ -506,7 +506,7 @@ config :pleroma, :rate_limit, Means that: 1. In 60 seconds, 15 authentication attempts can be performed from the same IP address. -2. In 1 second, 10 search requests can be performed from the same IP adress by unauthenticated users, while authenticated users can perform 30 search requests per second. +2. In 1 second, 10 search requests can be performed from the same IP address by unauthenticated users, while authenticated users can perform 30 search requests per second. Supported rate limiters: @@ -1081,7 +1081,7 @@ config :pleroma, Pleroma.Formatter, ## :configurable_from_database -Boolean, enables/disables in-database configuration. Read [Transfering the config to/from the database](../administration/CLI_tasks/config.md) for more information. +Boolean, enables/disables in-database configuration. Read [Transferring the config to/from the database](../administration/CLI_tasks/config.md) for more information. ## :database_config_whitelist @@ -1142,7 +1142,7 @@ Control favicons for instances. !!! note Requires enabled email -* `:purge_after_days` an integer, remove backup achives after N days. +* `:purge_after_days` an integer, remove backup achieves after N days. * `:limit_days` an integer, limit user to export not more often than once per N days. * `:dir` a string with a path to backup temporary directory or `nil` to let Pleroma choose temporary directory in the following order: 1. the directory named by the TMPDIR environment variable diff --git a/docs/configuration/custom_emoji.md b/docs/configuration/custom_emoji.md index 1648840fd..19250cf80 100644 --- a/docs/configuration/custom_emoji.md +++ b/docs/configuration/custom_emoji.md @@ -29,7 +29,7 @@ foo, /emoji/custom/foo.png The files should be PNG (APNG is okay with `.png` for `image/png` Content-type) and under 50kb for compatibility with mastodon. -Default file extentions and locations for emojis are set in `config.exs`. To use different locations or file-extentions, add the `shortcode_globs` to your secrets file (`prod.secret.exs` or `dev.secret.exs`) and edit it. Note that not all fediverse-software will show emojis with other file extentions: +Default file extensions and locations for emojis are set in `config.exs`. To use different locations or file-extensions, add the `shortcode_globs` to your secrets file (`prod.secret.exs` or `dev.secret.exs`) and edit it. Note that not all fediverse-software will show emojis with other file extensions: ```elixir config :pleroma, :emoji, shortcode_globs: ["/emoji/custom/**/*.png", "/emoji/custom/**/*.gif"] ``` diff --git a/docs/configuration/i2p.md b/docs/configuration/i2p.md index 8c5207d67..17dd9b0cb 100644 --- a/docs/configuration/i2p.md +++ b/docs/configuration/i2p.md @@ -1,4 +1,4 @@ -# I2P Federation and Accessability +# I2P Federation and Accessibility This guide is going to focus on the Pleroma federation aspect. The actual installation is neatly explained in the official documentation, and more likely to remain up-to-date. It might be added to this guide if there will be a need for that. diff --git a/docs/configuration/onion_federation.md b/docs/configuration/onion_federation.md index 37673211a..8a8137251 100644 --- a/docs/configuration/onion_federation.md +++ b/docs/configuration/onion_federation.md @@ -29,7 +29,7 @@ HiddenServiceDir /var/lib/tor/pleroma_hidden_service/ HiddenServicePort 80 127.0.0.1:8099 HiddenServiceVersion 3 # Remove if Tor version is below 0.3 ( tor --version ) ``` -Restart Tor to generate an adress: +Restart Tor to generate an address: ``` systemctl restart tor@default.service ``` diff --git a/docs/configuration/optimizing_beam.md b/docs/configuration/optimizing_beam.md index e336bd36c..5e81cd003 100644 --- a/docs/configuration/optimizing_beam.md +++ b/docs/configuration/optimizing_beam.md @@ -1,6 +1,6 @@ # Optimizing the BEAM -Pleroma is built upon the Erlang/OTP VM known as BEAM. The BEAM VM is highly optimized for latency, but this has drawbacks in environments without dedicated hardware. One of the tricks used by the BEAM VM is [busy waiting](https://en.wikipedia.org/wiki/Busy_waiting). This allows the application to pretend to be busy working so the OS kernel does not pause the application process and switch to another process waiting for the CPU to execute its workload. It does this by spinning for a period of time which inflates the apparent CPU usage of the application so it is immediately ready to execute another task. This can be observed with utilities like **top(1)** which will show consistently high CPU usage for the process. Switching between procesess is a rather expensive operation and also clears CPU caches further affecting latency and performance. The goal of busy waiting is to avoid this penalty. +Pleroma is built upon the Erlang/OTP VM known as BEAM. The BEAM VM is highly optimized for latency, but this has drawbacks in environments without dedicated hardware. One of the tricks used by the BEAM VM is [busy waiting](https://en.wikipedia.org/wiki/Busy_waiting). This allows the application to pretend to be busy working so the OS kernel does not pause the application process and switch to another process waiting for the CPU to execute its workload. It does this by spinning for a period of time which inflates the apparent CPU usage of the application so it is immediately ready to execute another task. This can be observed with utilities like **top(1)** which will show consistently high CPU usage for the process. Switching between processes is a rather expensive operation and also clears CPU caches further affecting latency and performance. The goal of busy waiting is to avoid this penalty. This strategy is very successful in making a performant and responsive application, but is not desirable on Virtual Machines or hardware with few CPU cores. Pleroma instances are often deployed on the same server as the required PostgreSQL database which can lead to situations where the Pleroma application is holding the CPU in a busy-wait loop and as a result the database cannot process requests in a timely manner. The fewer CPUs available, the more this problem is exacerbated. The latency is further amplified by the OS being installed on a Virtual Machine as the Hypervisor uses CPU time-slicing to pause the entire OS and switch between other tasks. diff --git a/docs/configuration/postgresql.md b/docs/configuration/postgresql.md index e251eb83b..56f1c60dc 100644 --- a/docs/configuration/postgresql.md +++ b/docs/configuration/postgresql.md @@ -22,7 +22,7 @@ config :pleroma, Pleroma.Repo, ] ``` -A more detailed explaination of the issue can be found at . +A more detailed explanation of the issue can be found at . ## Example configurations diff --git a/docs/configuration/search.md b/docs/configuration/search.md index f131948a7..0316c9bf4 100644 --- a/docs/configuration/search.md +++ b/docs/configuration/search.md @@ -38,7 +38,7 @@ indexes faster when it can process many posts in a single batch. Information about setting up meilisearch can be found in the [official documentation](https://docs.meilisearch.com/learn/getting_started/installation.html). You probably want to start it with `MEILI_NO_ANALYTICS=true` environment variable to disable analytics. -At least version 0.25.0 is required, but you are strongly adviced to use at least 0.26.0, as it introduces +At least version 0.25.0 is required, but you are strongly advised to use at least 0.26.0, as it introduces the `--enable-auto-batching` option which drastically improves performance. Without this option, the search is hardly usable on a somewhat big instance. @@ -61,7 +61,7 @@ You will see a "Default Admin API Key", this is the key you actually put into yo ### Initial indexing -After setting up the configuration, you'll want to index all of your already existsing posts. Only public posts are indexed. You'll only +After setting up the configuration, you'll want to index all of your already existing posts. Only public posts are indexed. You'll only have to do it one time, but it might take a while, depending on the amount of posts your instance has seen. This is also a fairly RAM consuming process for `meilisearch`, and it will take a lot of RAM when running if you have a lot of posts (seems to be around 5G for ~1.2 million posts while idle and up to 7G while indexing initially, but your experience may be different). -- cgit v1.2.3 From 9cfa4e67b11e5a1a7d09330581383dc67fcf6fd8 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?marcin=20miko=C5=82ajczak?= Date: Fri, 1 Mar 2024 18:14:31 +0100 Subject: Add ForceMention mrf MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: marcin mikołajczak --- docs/configuration/cheatsheet.md | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) (limited to 'docs/configuration') diff --git a/docs/configuration/cheatsheet.md b/docs/configuration/cheatsheet.md index 7bba7b26e..89a461b47 100644 --- a/docs/configuration/cheatsheet.md +++ b/docs/configuration/cheatsheet.md @@ -161,7 +161,8 @@ To add configuration to your config file, you can copy it from the base config. * `Pleroma.Web.ActivityPub.MRF.KeywordPolicy`: Rejects or removes from the federated timeline or replaces keywords. (See [`:mrf_keyword`](#mrf_keyword)). * `Pleroma.Web.ActivityPub.MRF.ForceMentionsInContent`: Forces every mentioned user to be reflected in the post content. * `Pleroma.Web.ActivityPub.MRF.InlineQuotePolicy`: Forces quote post URLs to be reflected in the message content inline. - * `Pleroma.Web.ActivityPub.MRF.QuoteToLinkTagPolicy`: Force a Link tag for posts quoting another post. (may break outgoing federation of quote posts with older Pleroma versions) + * `Pleroma.Web.ActivityPub.MRF.QuoteToLinkTagPolicy`: Force a Link tag for posts quoting another post. (may break outgoing federation of quote posts with older Pleroma versions). + * `Pleroma.Web.ActivityPub.MRF.ForceMention`: Forces posts to include a mention of the author of parent post or the author of quoted post. * `transparency`: Make the content of your Message Rewrite Facility settings public (via nodeinfo). * `transparency_exclusions`: Exclude specific instance names from MRF transparency. The use of the exclusions feature will be disclosed in nodeinfo as a boolean value. @@ -272,6 +273,10 @@ Notes: #### :mrf_inline_quote * `template`: The template to append to the post. `{url}` will be replaced with the actual link to the quoted post. Default: `RT: {url}` +#### :mrf_force_mention +* `mention_parent`: Whether to append mention of parent post author +* `mention_quoted`: Whether to append mention of parent quoted author + ### :activitypub * `unfollow_blocked`: Whether blocks result in people getting unfollowed * `outgoing_blocks`: Whether to federate blocks to other instances -- cgit v1.2.3 From c50f0f31f418037063bd97efcdc0f60b89594212 Mon Sep 17 00:00:00 2001 From: Lain Soykaf Date: Tue, 14 May 2024 16:56:58 +0400 Subject: Docs/Search: Add basic documentation of the qdrant search --- docs/configuration/search.md | 6 ++++++ 1 file changed, 6 insertions(+) (limited to 'docs/configuration') diff --git a/docs/configuration/search.md b/docs/configuration/search.md index 0316c9bf4..682d1e52a 100644 --- a/docs/configuration/search.md +++ b/docs/configuration/search.md @@ -10,6 +10,12 @@ To use built-in search that has no external dependencies, set the search module While it has no external dependencies, it has problems with performance and relevancy. +## QdrantSearch + +This uses the vector search engine [Qdrant](https://qdrant.tech) to search the posts in a vector space. This needs a way to generate embeddings, for now only the [Ollama](Ollama) api is supported. + +The default settings will support a setup where both Ollama and Qdrant run on the same system as pleroma. The embedding model used by Ollama will need to be pulled first (e.g. `ollama pull snowflake-arctic-embed:xs`) for the embedding to work. + ## Meilisearch Note that it's quite a bit more memory hungry than PostgreSQL (around 4-5G for ~1.2 million -- cgit v1.2.3 From e142ea400a9ed3595f8d432edd90ea26fc7d2eb5 Mon Sep 17 00:00:00 2001 From: Lain Soykaf Date: Sun, 19 May 2024 12:42:08 +0400 Subject: Docs: Switch docs from Ollama to OpenAI. --- docs/configuration/search.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) (limited to 'docs/configuration') diff --git a/docs/configuration/search.md b/docs/configuration/search.md index 682d1e52a..388f5acd1 100644 --- a/docs/configuration/search.md +++ b/docs/configuration/search.md @@ -12,9 +12,9 @@ While it has no external dependencies, it has problems with performance and rele ## QdrantSearch -This uses the vector search engine [Qdrant](https://qdrant.tech) to search the posts in a vector space. This needs a way to generate embeddings, for now only the [Ollama](Ollama) api is supported. +This uses the vector search engine [Qdrant](https://qdrant.tech) to search the posts in a vector space. This needs a way to generate embeddings and uses the [OpenAI API](https://platform.openai.com/docs/guides/embeddings/what-are-embeddings). This is implemented by several project besides OpenAI itself, including the python-based fastembed-server found in `supplemental/search/fastembed-api`. -The default settings will support a setup where both Ollama and Qdrant run on the same system as pleroma. The embedding model used by Ollama will need to be pulled first (e.g. `ollama pull snowflake-arctic-embed:xs`) for the embedding to work. +The default settings will support a setup where both the fastembed server and Qdrant run on the same system as pleroma. ## Meilisearch -- cgit v1.2.3 From 6a3a0cc0f5995185428c92f3c53e9c8524ea6856 Mon Sep 17 00:00:00 2001 From: Lain Soykaf Date: Sun, 19 May 2024 13:20:37 +0400 Subject: Docs: Write docs for the QdrantSearch --- docs/configuration/search.md | 20 +++++++++++++++++++- 1 file changed, 19 insertions(+), 1 deletion(-) (limited to 'docs/configuration') diff --git a/docs/configuration/search.md b/docs/configuration/search.md index 388f5acd1..6598e533f 100644 --- a/docs/configuration/search.md +++ b/docs/configuration/search.md @@ -14,7 +14,25 @@ While it has no external dependencies, it has problems with performance and rele This uses the vector search engine [Qdrant](https://qdrant.tech) to search the posts in a vector space. This needs a way to generate embeddings and uses the [OpenAI API](https://platform.openai.com/docs/guides/embeddings/what-are-embeddings). This is implemented by several project besides OpenAI itself, including the python-based fastembed-server found in `supplemental/search/fastembed-api`. -The default settings will support a setup where both the fastembed server and Qdrant run on the same system as pleroma. +The default settings will support a setup where both the fastembed server and Qdrant run on the same system as pleroma. To use it, set the search provider and run the fastembed server, see the README in `supplemental/search/fastembed-api`: + +https://qdrant.github.io/fastembed/examples/Supported_Models/ + +> config :pleroma, Pleroma.Search, module: Pleroma.Search.QdrantSearch + +You will also need to create the Qdrant index once by running `mix pleroma.search.indexer create_index`. Running `mix pleroma.search.indexer index` will retroactively index the last 100_000 activities. + +### Indexing and model options + +To see the available configuration options, check out the QdrantSearch section in `config/config.exs`. + +The default indexing option work for the default model (`snowflake-arctic-embed-xs`). To optimize for a low memory footprint, adjust the index configuration as described in the [Qdrant docs](https://qdrant.tech/documentation/guides/optimize/). + +Different embedding models will need different vector size settings. You can see a list of the models supported by the fastembed server [here](https://qdrant.github.io/fastembed/examples/Supported_Models), including their vector dimensions. These vector dimensions need to be set in the `qdrant_index_configuration`. + +E.g, If you want to use `sentence-transformers/all-MiniLM-L6-v2` as a model, you will not need to adjust things, because it and `snowflake-arctic-embed-xs` are both 384 dimensional models. If you want to use `snowflake/snowflake-arctic-embed-l`, you will need to adjust the `size` parameter in the `qdrant_index_configuration` to 1024, as it has a dimension of 1024. + +When using a different model, you will need do drop the index and recreate it (`mix pleroma.search.indexer drop_index` and `mix pleroma.search.indexer create_index`), as the different embeddings are not compatible with each other. ## Meilisearch -- cgit v1.2.3 From 6ec306d0684f3c5c05d768a3c431008925f21f15 Mon Sep 17 00:00:00 2001 From: Lain Soykaf Date: Sun, 19 May 2024 13:24:24 +0400 Subject: Docs: Add more information about index memory consumption. --- docs/configuration/search.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'docs/configuration') diff --git a/docs/configuration/search.md b/docs/configuration/search.md index 6598e533f..ed85acd2a 100644 --- a/docs/configuration/search.md +++ b/docs/configuration/search.md @@ -26,7 +26,7 @@ You will also need to create the Qdrant index once by running `mix pleroma.searc To see the available configuration options, check out the QdrantSearch section in `config/config.exs`. -The default indexing option work for the default model (`snowflake-arctic-embed-xs`). To optimize for a low memory footprint, adjust the index configuration as described in the [Qdrant docs](https://qdrant.tech/documentation/guides/optimize/). +The default indexing option work for the default model (`snowflake-arctic-embed-xs`). To optimize for a low memory footprint, adjust the index configuration as described in the [Qdrant docs](https://qdrant.tech/documentation/guides/optimize/). See also [this blog post](https://qdrant.tech/articles/memory-consumption/) that goes into detail. Different embedding models will need different vector size settings. You can see a list of the models supported by the fastembed server [here](https://qdrant.github.io/fastembed/examples/Supported_Models), including their vector dimensions. These vector dimensions need to be set in the `qdrant_index_configuration`. -- cgit v1.2.3 From dbaab6f54e306e5fb930ce1ed0699631c8aeaae1 Mon Sep 17 00:00:00 2001 From: Lain Soykaf Date: Sun, 19 May 2024 13:38:31 +0400 Subject: Docs: Mention running the Qdrant server --- docs/configuration/search.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) (limited to 'docs/configuration') diff --git a/docs/configuration/search.md b/docs/configuration/search.md index ed85acd2a..d34f84d4f 100644 --- a/docs/configuration/search.md +++ b/docs/configuration/search.md @@ -16,10 +16,10 @@ This uses the vector search engine [Qdrant](https://qdrant.tech) to search the p The default settings will support a setup where both the fastembed server and Qdrant run on the same system as pleroma. To use it, set the search provider and run the fastembed server, see the README in `supplemental/search/fastembed-api`: -https://qdrant.github.io/fastembed/examples/Supported_Models/ - > config :pleroma, Pleroma.Search, module: Pleroma.Search.QdrantSearch +Then, start the Qdrant server, see [here](https://qdrant.tech/documentation/quick-start/) for instructions. + You will also need to create the Qdrant index once by running `mix pleroma.search.indexer create_index`. Running `mix pleroma.search.indexer index` will retroactively index the last 100_000 activities. ### Indexing and model options -- cgit v1.2.3 From fc7ce339edc40cb791d321a20f01f2568337b845 Mon Sep 17 00:00:00 2001 From: Lain Soykaf Date: Mon, 27 May 2024 21:28:20 +0400 Subject: Cheatsheet: Add allow_unsafe_eval --- docs/configuration/cheatsheet.md | 1 + 1 file changed, 1 insertion(+) (limited to 'docs/configuration') diff --git a/docs/configuration/cheatsheet.md b/docs/configuration/cheatsheet.md index ca2ce6369..78997c4db 100644 --- a/docs/configuration/cheatsheet.md +++ b/docs/configuration/cheatsheet.md @@ -472,6 +472,7 @@ This will make Pleroma listen on `127.0.0.1` port `8080` and generate urls start * ``ct_max_age``: The maximum age for the `Expect-CT` header if sent. * ``referrer_policy``: The referrer policy to use, either `"same-origin"` or `"no-referrer"`. * ``report_uri``: Adds the specified url to `report-uri` and `report-to` group in CSP header. +* `allow_unsafe_eval`: Adds `wasm-unsafe-eval` to the CSP header. Needed for some non-essential frontend features like Flash emulation. ### Pleroma.Web.Plugs.RemoteIp -- cgit v1.2.3