diff options
67 files changed, 2235 insertions, 100 deletions
diff --git a/CHANGELOG.md b/CHANGELOG.md index 6a299804c..f1beb0cd0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -12,6 +12,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/). ### Changed - Allow users to remove their emails if instance does not need email to register +- Uploadfilter `Pleroma.Upload.Filter.Exiftool` has been renamed to `Pleroma.Upload.Filter.Exiftool.StripLocation` ### Added - `activeMonth` and `activeHalfyear` fields in NodeInfo usage.users object @@ -30,6 +31,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/). - Configuration: Add `birthday_required` and `birthday_min_age` settings to provide a way to require users to enter their birth date. - PleromaAPI: Add `GET /api/v1/pleroma/birthdays` API endpoint - Make backend-rendered pages translatable. This includes emails. Pages returned as a HTTP response are translated using the language specified in the `userLanguage` cookie, or the `Accept-Language` header. Emails are translated using the `language` field when registering. This language can be changed by `PATCH /api/v1/accounts/update_credentials` with the `language` field. +- Uploadfilter `Pleroma.Upload.Filter.Exiftool.ReadDescription` returns description values to the FE so they can pre fill the image description field ### Fixed - Subscription(Bell) Notifications: Don't create from Pipeline Ingested replies @@ -48,6 +50,12 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/). ### Removed +## 2.4.3 - 2022-05-06 + +### Security +- Private `/objects/` and `/activities/` leaking if cached by authenticated user +- SweetXML library DTD bomb + ## 2.4.2 - 2022-01-10 ### Fixed @@ -91,6 +99,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/). - Improved Twittercard and OpenGraph meta tag generation including thumbnails and image dimension metadata when available. - AdminAPI: sort users so the newest are at the top. - ActivityPub Client-to-Server(C2S): Limitation on the type of Activity/Object are lifted as they are now passed through ObjectValidators +- MRF (`AntiFollowbotPolicy`): Bot accounts are now also considered followbots. Users can still allow bots to follow them by first following the bot. ### Added diff --git a/config/description.exs b/config/description.exs index 06b0182cd..b29348edf 100644 --- a/config/description.exs +++ b/config/description.exs @@ -2735,7 +2735,7 @@ config :pleroma, :config_description, [ key: :versions, type: {:list, :atom}, description: "List of TLS version to use", - suggestions: [:tlsv1, ":tlsv1.1", ":tlsv1.2"] + suggestions: [:tlsv1, ":tlsv1.1", ":tlsv1.2", ":tlsv1.3"] } ] } diff --git a/docs/administration/CLI_tasks/instance.md b/docs/administration/CLI_tasks/instance.md index 982b22bf3..88509cf5b 100644 --- a/docs/administration/CLI_tasks/instance.md +++ b/docs/administration/CLI_tasks/instance.md @@ -37,7 +37,8 @@ If any of the options are left unspecified, you will be prompted interactively. - `--static-dir <path>` - the directory custom public files should be read from (custom emojis, frontend bundle overrides, robots.txt, etc.) - `--listen-ip <ip>` - the ip the app should listen to, defaults to 127.0.0.1 - `--listen-port <port>` - the port the app should listen to, defaults to 4000 -- `--strip-uploads <Y|N>` - use ExifTool to strip uploads of sensitive location data +- `--strip-uploads-location <Y|N>` - use ExifTool to strip uploads of sensitive location data +- `--read-uploads-description <Y|N>` - use ExifTool to read image descriptions from uploads - `--anonymize-uploads <Y|N>` - randomize uploaded filenames - `--dedupe-uploads <Y|N>` - store files based on their hash to reduce data storage requirements if duplicates are uploaded with different filenames - `--skip-release-env` - skip generation the release environment file diff --git a/docs/administration/updating.md b/docs/administration/updating.md index ef2c9218c..01d3b9b0e 100644 --- a/docs/administration/updating.md +++ b/docs/administration/updating.md @@ -17,11 +17,11 @@ su pleroma -s $SHELL -lc "./bin/pleroma_ctl migrate" ## For from source installations (using git) 1. Go to the working directory of Pleroma (default is `/opt/pleroma`) -2. Run `git pull`. This pulls the latest changes from upstream. +2. Run `git pull` [^1]. This pulls the latest changes from upstream. 3. Run `mix deps.get` [^1]. This pulls in any new dependencies. 4. Stop the Pleroma service. 5. Run `mix ecto.migrate` [^1] [^2]. This task performs database migrations, if there were any. 6. Start the Pleroma service. -[^1]: Depending on which install guide you followed (for example on Debian/Ubuntu), you want to run `mix` tasks as `pleroma` user by adding `sudo -Hu pleroma` before the command. +[^1]: Depending on which install guide you followed (for example on Debian/Ubuntu), you want to run `git` and `mix` tasks as `pleroma` user by adding `sudo -Hu pleroma` before the command. [^2]: Prefix with `MIX_ENV=prod` to run it using the production config file. diff --git a/docs/configuration/cheatsheet.md b/docs/configuration/cheatsheet.md index 0ab600b21..6e13b9622 100644 --- a/docs/configuration/cheatsheet.md +++ b/docs/configuration/cheatsheet.md @@ -126,6 +126,7 @@ To add configuration to your config file, you can copy it from the base config. * `Pleroma.Web.ActivityPub.MRF.ActivityExpirationPolicy`: Sets a default expiration on all posts made by users of the local instance. Requires `Pleroma.Workers.PurgeExpiredActivity` to be enabled for processing the scheduled delections. * `Pleroma.Web.ActivityPub.MRF.ForceBotUnlistedPolicy`: Makes all bot posts to disappear from public timelines. * `Pleroma.Web.ActivityPub.MRF.FollowBotPolicy`: Automatically follows newly discovered users from the specified bot account. Local accounts, locked accounts, and users with "#nobot" in their bio are respected and excluded from being followed. + * `Pleroma.Web.ActivityPub.MRF.AntiFollowbotPolicy`: Drops follow requests from followbots. Users can still allow bots to follow them by first following the bot. * `Pleroma.Web.ActivityPub.MRF.KeywordPolicy`: Rejects or removes from the federated timeline or replaces keywords. (See [`:mrf_keyword`](#mrf_keyword)). * `Pleroma.Web.ActivityPub.MRF.ForceMentionsInContent`: Forces every mentioned user to be reflected in the post content. * `transparency`: Make the content of your Message Rewrite Facility settings public (via nodeinfo). @@ -627,12 +628,18 @@ This filter replaces the filename (not the path) of an upload. For complete obfu No specific configuration. -#### Pleroma.Upload.Filter.Exiftool +#### Pleroma.Upload.Filter.Exiftool.StripLocation This filter only strips the GPS and location metadata with Exiftool leaving color profiles and attributes intact. No specific configuration. +#### Pleroma.Upload.Filter.Exiftool.ReadDescription + +This filter reads the ImageDescription and iptc:Caption-Abstract fields with Exiftool so clients can prefill the media description field. + +No specific configuration. + #### Pleroma.Upload.Filter.Mogrify * `args`: List of actions for the `mogrify` command like `"strip"` or `["strip", "auto-orient", {"implode", "1"}]`. diff --git a/docs/development/API/admin_api.md b/docs/development/API/admin_api.md index f14081893..c46f83839 100644 --- a/docs/development/API/admin_api.md +++ b/docs/development/API/admin_api.md @@ -1636,3 +1636,117 @@ Returns the content of the document "error": "Could not install frontend" } ``` + +## `GET /api/v1/pleroma/admin/announcements` + +### List announcements + +- Params: `offset`, `limit` + +- Response: JSON, list of announcements + +```json +[ + { + "id": "AHDp0GBdRn1EPN5HN2", + "content": "some content", + "starts_at": null, + "ends_at": null, + "all_day": false, + "published_at": "2022-03-09T02:13:05", + "reactions": [], + "statuses": [], + "tags": [], + "emojis": [], + "updated_at": "2022-03-09T02:13:05" + } +] +``` + +Note that this differs from the Mastodon API variant: Mastodon API only returns *active* announcements, while this returns all. + +## `GET /api/v1/pleroma/admin/announcements/:id` + +### Display one announcement + +- Response: JSON, one announcement + +```json +{ + "id": "AHDp0GBdRn1EPN5HN2", + "content": "some content", + "starts_at": null, + "ends_at": null, + "all_day": false, + "published_at": "2022-03-09T02:13:05", + "reactions": [], + "statuses": [], + "tags": [], + "emojis": [], + "updated_at": "2022-03-09T02:13:05" +} +``` + +## `POST /api/v1/pleroma/admin/announcements` + +### Create an announcement + +- Params: + - `content`: string, required, announcement content + - `starts_at`: datetime, optional, default to null, the time when the announcement will become active (displayed to users); if it is null, the announcement will be active immediately + - `ends_at`: datetime, optional, default to null, the time when the announcement will become inactive (no longer displayed to users); if it is null, the announcement will be active until an admin deletes it + - `all_day`: boolean, optional, default to false, tells the client whether to only display dates for `starts_at` and `ends_at` + +- Response: JSON, created announcement + +```json +{ + "id": "AHDp0GBdRn1EPN5HN2", + "content": "some content", + "starts_at": null, + "ends_at": null, + "all_day": false, + "published_at": "2022-03-09T02:13:05", + "reactions": [], + "statuses": [], + "tags": [], + "emojis": [], + "updated_at": "2022-03-09T02:13:05" +} +``` + +## `PATCH /api/v1/pleroma/admin/announcements/:id` + +### Change an announcement + +- Params: same as `POST /api/v1/pleroma/admin/announcements`, except no param is required. + +- Updates the announcement according to params. Missing params are kept as-is. + +- Response: JSON, updated announcement + +```json +{ + "id": "AHDp0GBdRn1EPN5HN2", + "content": "some content", + "starts_at": null, + "ends_at": null, + "all_day": false, + "published_at": "2022-03-09T02:13:05", + "reactions": [], + "statuses": [], + "tags": [], + "emojis": [], + "updated_at": "2022-03-09T02:13:05" +} +``` + +## `DELETE /api/v1/pleroma/admin/announcements/:id` + +### Delete an announcement + +- Response: JSON, empty object + +```json +{} +``` diff --git a/docs/installation/optional/media_graphics_packages.md b/docs/installation/optional/media_graphics_packages.md index cb3d71188..de402d1c4 100644 --- a/docs/installation/optional/media_graphics_packages.md +++ b/docs/installation/optional/media_graphics_packages.md @@ -1,9 +1,9 @@ # Optional software packages needed for specific functionality For specific Pleroma functionality (which is disabled by default) some or all of the below packages are required: - * `ImageMagic` - * `ffmpeg` - * `exiftool` + * `ImageMagic` + * `ffmpeg` + * `exiftool` Please refer to documentation in `docs/installation` on how to install them on specific OS. @@ -14,19 +14,20 @@ Note: the packages are not required with the current default settings of Pleroma `ImageMagick` is a set of tools to create, edit, compose, or convert bitmap images. It is required for the following Pleroma features: - * `Pleroma.Upload.Filters.Mogrify`, `Pleroma.Upload.Filters.Mogrifun` upload filters (related config: `Plaroma.Upload/filters` in `config/config.exs`) - * Media preview proxy for still images (related config: `media_preview_proxy/enabled` in `config/config.exs`) + * `Pleroma.Upload.Filters.Mogrify`, `Pleroma.Upload.Filters.Mogrifun` upload filters (related config: `Plaroma.Upload/filters` in `config/config.exs`) + * Media preview proxy for still images (related config: `media_preview_proxy/enabled` in `config/config.exs`) ## `ffmpeg` `ffmpeg` is software to record, convert and stream audio and video. It is required for the following Pleroma features: - * Media preview proxy for videos (related config: `media_preview_proxy/enabled` in `config/config.exs`) + * Media preview proxy for videos (related config: `media_preview_proxy/enabled` in `config/config.exs`) ## `exiftool` `exiftool` is media files metadata reader/writer. It is required for the following Pleroma features: - * `Pleroma.Upload.Filters.Exiftool` upload filter (related config: `Plaroma.Upload/filters` in `config/config.exs`) + * `Pleroma.Upload.Filters.Exiftool.StripLocation` upload filter (related config: `Plaroma.Upload/filters` in `config/config.exs`) + * `Pleroma.Upload.Filters.Exiftool.ReadDescription` upload filter (related config: `Plaroma.Upload/filters` in `config/config.exs`) diff --git a/lib/mix/tasks/pleroma/instance.ex b/lib/mix/tasks/pleroma/instance.ex index f292fc762..5c93f19ff 100644 --- a/lib/mix/tasks/pleroma/instance.ex +++ b/lib/mix/tasks/pleroma/instance.ex @@ -34,7 +34,8 @@ defmodule Mix.Tasks.Pleroma.Instance do static_dir: :string, listen_ip: :string, listen_port: :string, - strip_uploads: :string, + strip_uploads_location: :string, + read_uploads_description: :string, anonymize_uploads: :string, dedupe_uploads: :string ], @@ -161,7 +162,7 @@ defmodule Mix.Tasks.Pleroma.Instance do ) |> Path.expand() - {strip_uploads_message, strip_uploads_default} = + {strip_uploads_location_message, strip_uploads_location_default} = if Pleroma.Utils.command_available?("exiftool") do {"Do you want to strip location (GPS) data from uploaded images? This requires exiftool, it was detected as installed. (y/n)", "y"} @@ -170,12 +171,29 @@ defmodule Mix.Tasks.Pleroma.Instance do "n"} end - strip_uploads = + strip_uploads_location = get_option( options, - :strip_uploads, - strip_uploads_message, - strip_uploads_default + :strip_uploads_location, + strip_uploads_location_message, + strip_uploads_location_default + ) === "y" + + {read_uploads_description_message, read_uploads_description_default} = + if Pleroma.Utils.command_available?("exiftool") do + {"Do you want to read data from uploaded files so clients can use it to prefill fields like image description? This requires exiftool, it was detected as installed. (y/n)", + "y"} + else + {"Do you want to read data from uploaded files so clients can use it to prefill fields like image description? This requires exiftool, it was detected as not installed, please install it if you answer yes. (y/n)", + "n"} + end + + read_uploads_description = + get_option( + options, + :read_uploads_description, + read_uploads_description_message, + read_uploads_description_default ) === "y" anonymize_uploads = @@ -229,7 +247,8 @@ defmodule Mix.Tasks.Pleroma.Instance do listen_port: listen_port, upload_filters: upload_filters(%{ - strip: strip_uploads, + strip_location: strip_uploads_location, + read_description: read_uploads_description, anonymize: anonymize_uploads, dedupe: dedupe_uploads }) @@ -297,13 +316,20 @@ defmodule Mix.Tasks.Pleroma.Instance do defp upload_filters(filters) when is_map(filters) do enabled_filters = - if filters.strip do - [Pleroma.Upload.Filter.Exiftool] + if filters.strip_location do + [Pleroma.Upload.Filter.Exiftool.StripLocation] else [] end enabled_filters = + if filters.read_description do + enabled_filters ++ [Pleroma.Upload.Filter.Exiftool.ReadDescription] + else + enabled_filters + end + + enabled_filters = if filters.anonymize do enabled_filters ++ [Pleroma.Upload.Filter.AnonymizeFilename] else diff --git a/lib/pleroma/announcement.ex b/lib/pleroma/announcement.ex new file mode 100644 index 000000000..d97c5e728 --- /dev/null +++ b/lib/pleroma/announcement.ex @@ -0,0 +1,160 @@ +# Pleroma: A lightweight social networking server +# Copyright © 2017-2022 Pleroma Authors <https://pleroma.social/> +# SPDX-License-Identifier: AGPL-3.0-only + +defmodule Pleroma.Announcement do + use Ecto.Schema + + import Ecto.Changeset, only: [cast: 3, validate_required: 2] + import Ecto.Query + + alias Pleroma.AnnouncementReadRelationship + alias Pleroma.Repo + + @type t :: %__MODULE__{} + @primary_key {:id, FlakeId.Ecto.CompatType, autogenerate: true} + + schema "announcements" do + field(:data, :map) + field(:starts_at, :utc_datetime) + field(:ends_at, :utc_datetime) + field(:rendered, :map) + + timestamps(type: :utc_datetime) + end + + def change(struct, params \\ %{}) do + struct + |> cast(validate_params(struct, params), [:data, :starts_at, :ends_at, :rendered]) + |> validate_required([:data]) + end + + defp validate_params(struct, params) do + base_data = + %{ + "content" => "", + "all_day" => false + } + |> Map.merge((struct && struct.data) || %{}) + + merged_data = + Map.merge(base_data, params.data) + |> Map.take(["content", "all_day"]) + + params + |> Map.merge(%{data: merged_data}) + |> add_rendered_properties() + end + + def add_rendered_properties(params) do + {content_html, _, _} = + Pleroma.Web.CommonAPI.Utils.format_input(params.data["content"], "text/plain", + mentions_format: :full + ) + + rendered = %{ + "content" => content_html + } + + params + |> Map.put(:rendered, rendered) + end + + def add(params) do + changeset = change(%__MODULE__{}, params) + + Repo.insert(changeset) + end + + def update(announcement, params) do + changeset = change(announcement, params) + + Repo.update(changeset) + end + + def list_all do + __MODULE__ + |> Repo.all() + end + + def list_paginated(%{limit: limited_number, offset: offset_number}) do + __MODULE__ + |> limit(^limited_number) + |> offset(^offset_number) + |> Repo.all() + end + + def get_by_id(id) do + Repo.get_by(__MODULE__, id: id) + end + + def delete_by_id(id) do + with announcement when not is_nil(announcement) <- get_by_id(id), + {:ok, _} <- Repo.delete(announcement) do + :ok + else + _ -> + :error + end + end + + def read_by?(announcement, user) do + AnnouncementReadRelationship.exists?(user, announcement) + end + + def mark_read_by(announcement, user) do + AnnouncementReadRelationship.mark_read(user, announcement) + end + + def render_json(announcement, opts \\ []) do + extra_params = + case Keyword.fetch(opts, :for) do + {:ok, user} when not is_nil(user) -> + %{read: read_by?(announcement, user)} + + _ -> + %{} + end + + admin_extra_params = + case Keyword.fetch(opts, :admin) do + {:ok, true} -> + %{pleroma: %{raw_content: announcement.data["content"]}} + + _ -> + %{} + end + + base = %{ + id: announcement.id, + content: announcement.rendered["content"], + starts_at: announcement.starts_at, + ends_at: announcement.ends_at, + all_day: announcement.data["all_day"], + published_at: announcement.inserted_at, + updated_at: announcement.updated_at, + mentions: [], + statuses: [], + tags: [], + emojis: [], + reactions: [] + } + + base + |> Map.merge(extra_params) + |> Map.merge(admin_extra_params) + end + + # "visible" means: + # starts_at < time < ends_at + def list_all_visible_when(time) do + __MODULE__ + |> where([a], is_nil(a.starts_at) or a.starts_at < ^time) + |> where([a], is_nil(a.ends_at) or a.ends_at > ^time) + |> Repo.all() + end + + def list_all_visible do + list_all_visible_when(DateTime.now("Etc/UTC") |> elem(1)) + end +end diff --git a/lib/pleroma/announcement_read_relationship.ex b/lib/pleroma/announcement_read_relationship.ex new file mode 100644 index 000000000..9b64404ce --- /dev/null +++ b/lib/pleroma/announcement_read_relationship.ex @@ -0,0 +1,55 @@ +# Pleroma: A lightweight social networking server +# Copyright © 2017-2022 Pleroma Authors <https://pleroma.social/> +# SPDX-License-Identifier: AGPL-3.0-only + +defmodule Pleroma.AnnouncementReadRelationship do + use Ecto.Schema + + import Ecto.Changeset + + alias FlakeId.Ecto.CompatType + alias Pleroma.Announcement + alias Pleroma.Repo + alias Pleroma.User + + @type t :: %__MODULE__{} + + schema "announcement_read_relationships" do + belongs_to(:user, User, type: CompatType) + belongs_to(:announcement, Announcement, type: CompatType) + + timestamps(updated_at: false) + end + + def mark_read(user, announcement) do + %__MODULE__{} + |> cast(%{user_id: user.id, announcement_id: announcement.id}, [:user_id, :announcement_id]) + |> validate_required([:user_id, :announcement_id]) + |> foreign_key_constraint(:user_id) + |> foreign_key_constraint(:announcement_id) + |> unique_constraint([:user_id, :announcement_id]) + |> Repo.insert() + end + + def mark_unread(user, announcement) do + with relationship <- get(user, announcement), + {:exists, true} <- {:exists, not is_nil(relationship)}, + {:ok, _} <- Repo.delete(relationship) do + :ok + else + {:exists, false} -> + :ok + + _ -> + :error + end + end + + def get(user, announcement) do + Repo.get_by(__MODULE__, user_id: user.id, announcement_id: announcement.id) + end + + def exists?(user, announcement) do + not is_nil(get(user, announcement)) + end +end diff --git a/lib/pleroma/application_requirements.ex b/lib/pleroma/application_requirements.ex index 06d388694..44b1c1705 100644 --- a/lib/pleroma/application_requirements.ex +++ b/lib/pleroma/application_requirements.ex @@ -164,7 +164,8 @@ defmodule Pleroma.ApplicationRequirements do defp check_system_commands!(:ok) do filter_commands_statuses = [ - check_filter(Pleroma.Upload.Filter.Exiftool, "exiftool"), + check_filter(Pleroma.Upload.Filter.Exiftool.StripLocation, "exiftool"), + check_filter(Pleroma.Upload.Filter.Exiftool.ReadDescription, "exiftool"), check_filter(Pleroma.Upload.Filter.Mogrify, "mogrify"), check_filter(Pleroma.Upload.Filter.Mogrifun, "mogrify"), check_filter(Pleroma.Upload.Filter.AnalyzeMetadata, "mogrify"), diff --git a/lib/pleroma/config/deprecation_warnings.ex b/lib/pleroma/config/deprecation_warnings.ex index 118dd3acc..599f1d3cf 100644 --- a/lib/pleroma/config/deprecation_warnings.ex +++ b/lib/pleroma/config/deprecation_warnings.ex @@ -20,6 +20,43 @@ defmodule Pleroma.Config.DeprecationWarnings do "\n* `config :pleroma, :instance, mrf_transparency_exclusions` is now `config :pleroma, :mrf, transparency_exclusions`"} ] + def check_exiftool_filter do + filters = Config.get([Pleroma.Upload]) |> Keyword.get(:filters, []) + + if Pleroma.Upload.Filter.Exiftool in filters do + Logger.warn(""" + !!!DEPRECATION WARNING!!! + Your config is using Exiftool as a filter instead of Exiftool.StripLocation. This should work for now, but you are advised to change to the new configuration to prevent possible issues later: + + ``` + config :pleroma, Pleroma.Upload, + filters: [Pleroma.Upload.Filter.Exiftool] + ``` + + Is now + + + ``` + config :pleroma, Pleroma.Upload, + filters: [Pleroma.Upload.Filter.Exiftool.StripLocation] + ``` + """) + + new_config = + filters + |> Enum.map(fn + Pleroma.Upload.Filter.Exiftool -> Pleroma.Upload.Filter.Exiftool.StripLocation + filter -> filter + end) + + Config.put([Pleroma.Upload, :filters], new_config) + + :error + else + :ok + end + end + def check_simple_policy_tuples do has_strings = Config.get([:mrf_simple]) @@ -180,7 +217,8 @@ defmodule Pleroma.Config.DeprecationWarnings do check_old_chat_shoutbox(), check_quarantined_instances_tuples(), check_transparency_exclusions_tuples(), - check_simple_policy_tuples() + check_simple_policy_tuples(), + check_exiftool_filter() ] |> Enum.reduce(:ok, fn :ok, :ok -> :ok diff --git a/lib/pleroma/constants.ex b/lib/pleroma/constants.ex index a42c71d23..7b63ab06e 100644 --- a/lib/pleroma/constants.ex +++ b/lib/pleroma/constants.ex @@ -27,4 +27,10 @@ defmodule Pleroma.Constants do do: ~w(index.html robots.txt static static-fe finmoji emoji packs sounds images instance sw.js sw-pleroma.js favicon.png schemas doc embed.js embed.css) ) + + # basic regex, just there to weed out potential mistakes + # https://datatracker.ietf.org/doc/html/rfc2045#section-5.1 + const(mime_regex, + do: ~r/^[^[:cntrl:] ()<>@,;:\\"\/\[\]?=]+\/[^[:cntrl:] ()<>@,;:\\"\/\[\]?=]+(; .*)?$/ + ) end diff --git a/lib/pleroma/ecto_type/activity_pub/object_validators/mime.ex b/lib/pleroma/ecto_type/activity_pub/object_validators/mime.ex new file mode 100644 index 000000000..31d51577d --- /dev/null +++ b/lib/pleroma/ecto_type/activity_pub/object_validators/mime.ex @@ -0,0 +1,25 @@ +# Pleroma: A lightweight social networking server +# Copyright © 2017-2022 Pleroma Authors <https://pleroma.social/> +# SPDX-License-Identifier: AGPL-3.0-only + +defmodule Pleroma.EctoType.ActivityPub.ObjectValidators.MIME do + use Ecto.Type + + require Pleroma.Constants + + def type, do: :string + + def cast(mime) when is_binary(mime) do + if mime =~ Pleroma.Constants.mime_regex() do + {:ok, mime} + else + {:ok, "application/octet-stream"} + end + end + + def cast(_), do: :error + + def dump(data), do: {:ok, data} + + def load(data), do: {:ok, data} +end diff --git a/lib/pleroma/http/adapter_helper/hackney.ex b/lib/pleroma/http/adapter_helper/hackney.ex index b4f2f0cc2..f3be1f3d0 100644 --- a/lib/pleroma/http/adapter_helper/hackney.ex +++ b/lib/pleroma/http/adapter_helper/hackney.ex @@ -24,10 +24,6 @@ defmodule Pleroma.HTTP.AdapterHelper.Hackney do |> Pleroma.HTTP.AdapterHelper.maybe_add_proxy(proxy) end - defp add_scheme_opts(opts, %URI{scheme: "https"}) do - Keyword.put(opts, :ssl_options, versions: [:"tlsv1.2", :"tlsv1.1", :tlsv1]) - end - defp add_scheme_opts(opts, _), do: opts defp maybe_add_with_body(opts) do diff --git a/lib/pleroma/reverse_proxy/client/hackney.ex b/lib/pleroma/reverse_proxy/client/hackney.ex index 41eaf06cc..d3e986912 100644 --- a/lib/pleroma/reverse_proxy/client/hackney.ex +++ b/lib/pleroma/reverse_proxy/client/hackney.ex @@ -7,7 +7,6 @@ defmodule Pleroma.ReverseProxy.Client.Hackney do @impl true def request(method, url, headers, body, opts \\ []) do - opts = Keyword.put(opts, :ssl_options, versions: [:"tlsv1.2", :"tlsv1.1", :tlsv1]) :hackney.request(method, url, headers, body, opts) end diff --git a/lib/pleroma/upload.ex b/lib/pleroma/upload.ex index 242813dcd..db2909276 100644 --- a/lib/pleroma/upload.ex +++ b/lib/pleroma/upload.ex @@ -60,12 +60,23 @@ defmodule Pleroma.Upload do width: integer(), height: integer(), blurhash: String.t(), + description: String.t(), path: String.t() } - defstruct [:id, :name, :tempfile, :content_type, :width, :height, :blurhash, :path] - - defp get_description(opts, upload) do - case {opts[:description], Pleroma.Config.get([Pleroma.Upload, :default_description])} do + defstruct [ + :id, + :name, + :tempfile, + :content_type, + :width, + :height, + :blurhash, + :description, + :path + ] + + defp get_description(upload) do + case {upload.description, Pleroma.Config.get([Pleroma.Upload, :default_description])} do {description, _} when is_binary(description) -> description {_, :filename} -> upload.name {_, str} when is_binary(str) -> str @@ -81,7 +92,7 @@ defmodule Pleroma.Upload do with {:ok, upload} <- prepare_upload(upload, opts), upload = %__MODULE__{upload | path: upload.path || "#{upload.id}/#{upload.name}"}, {:ok, upload} <- Pleroma.Upload.Filter.filter(opts.filters, upload), - description = get_description(opts, upload), + description = get_description(upload), {_, true} <- {:description_limit, String.length(description) <= Pleroma.Config.get([:instance, :description_limit])}, @@ -152,7 +163,8 @@ defmodule Pleroma.Upload do id: UUID.generate(), name: file.filename, tempfile: file.path, - content_type: file.content_type + content_type: file.content_type, + description: opts.description }} end end @@ -172,7 +184,8 @@ defmodule Pleroma.Upload do id: UUID.generate(), name: hash <> "." <> ext, tempfile: tmp_path, - content_type: content_type + content_type: content_type, + description: opts.description }} end end diff --git a/lib/pleroma/upload/filter/exiftool/read_description.ex b/lib/pleroma/upload/filter/exiftool/read_description.ex new file mode 100644 index 000000000..03d698a81 --- /dev/null +++ b/lib/pleroma/upload/filter/exiftool/read_description.ex @@ -0,0 +1,49 @@ +# Pleroma: A lightweight social networking server +# Copyright © 2017-2021 Pleroma Authors <https://pleroma.social/> +# SPDX-License-Identifier: AGPL-3.0-only + +defmodule Pleroma.Upload.Filter.Exiftool.ReadDescription do + @moduledoc """ + Gets a valid description from the related EXIF tags and provides them in the response if no description is provided yet. + It will first check ImageDescription, when that doesn't probide a valid description, it will check iptc:Caption-Abstract. + A valid description means the fields are filled in and not too long (see `:instance, :description_limit`). + """ + @behaviour Pleroma.Upload.Filter + + @spec filter(Pleroma.Upload.t()) :: {:ok, any()} | {:error, String.t()} + + def filter(%Pleroma.Upload{description: description}) + when is_binary(description), + do: {:ok, :noop} + + def filter(%Pleroma.Upload{tempfile: file} = upload), + do: {:ok, :filtered, upload |> Map.put(:description, read_description_from_exif_data(file))} + + def filter(_, _), do: {:ok, :noop} + + defp read_description_from_exif_data(file) do + nil + |> read_when_empty(file, "-ImageDescription") + |> read_when_empty(file, "-iptc:Caption-Abstract") + end + + defp read_when_empty(current_description, _, _) when is_binary(current_description), + do: current_description + + defp read_when_empty(_, file, tag) do + try do + {tag_content, 0} = + System.cmd("exiftool", ["-b", "-s3", tag, file], stderr_to_stdout: true, parallelism: true) + + tag_content = String.trim(tag_content) + + if tag_content != "" and + String.length(tag_content) <= + Pleroma.Config.get([:instance, :description_limit]), + do: tag_content, + else: nil + rescue + _ in ErlangError -> nil + end + end +end diff --git a/lib/pleroma/upload/filter/exiftool.ex b/lib/pleroma/upload/filter/exiftool/strip_location.ex index 36cc045c2..6100527d3 100644 --- a/lib/pleroma/upload/filter/exiftool.ex +++ b/lib/pleroma/upload/filter/exiftool/strip_location.ex @@ -2,7 +2,7 @@ # Copyright © 2017-2022 Pleroma Authors <https://pleroma.social/> # SPDX-License-Identifier: AGPL-3.0-only -defmodule Pleroma.Upload.Filter.Exiftool do +defmodule Pleroma.Upload.Filter.Exiftool.StripLocation do @moduledoc """ Strips GPS related EXIF tags and overwrites the file in place. Also strips or replaces filesystem metadata e.g., timestamps. diff --git a/lib/pleroma/user.ex b/lib/pleroma/user.ex index 747a83e8d..712d3b1d9 100644 --- a/lib/pleroma/user.ex +++ b/lib/pleroma/user.ex @@ -706,7 +706,7 @@ defmodule Pleroma.User do ]) |> validate_required([:name, :nickname]) |> unique_constraint(:nickname) - |> validate_exclusion(:nickname, Config.get([User, :restricted_nicknames])) + |> validate_not_restricted_nickname(:nickname) |> validate_format(:nickname, local_nickname_regex()) |> put_ap_id() |> unique_constraint(:ap_id) @@ -754,17 +754,9 @@ defmodule Pleroma.User do |> validate_confirmation(:password) |> unique_constraint(:email) |> validate_format(:email, @email_regex) - |> validate_change(:email, fn :email, email -> - valid? = - Config.get([User, :email_blacklist]) - |> Enum.all?(fn blacklisted_domain -> - !String.ends_with?(email, ["@" <> blacklisted_domain, "." <> blacklisted_domain]) - end) - - if valid?, do: [], else: [email: "Invalid email"] - end) + |> validate_email_not_in_blacklisted_domain(:email) |> unique_constraint(:nickname) - |> validate_exclusion(:nickname, Config.get([User, :restricted_nicknames])) + |> validate_not_restricted_nickname(:nickname) |> validate_format(:nickname, local_nickname_regex()) |> validate_length(:bio, max: bio_limit) |> validate_length(:name, min: 1, max: name_limit) @@ -778,6 +770,35 @@ defmodule Pleroma.User do |> put_following_and_follower_and_featured_address() end + def validate_not_restricted_nickname(changeset, field) do + validate_change(changeset, field, fn _, value -> + valid? = + Config.get([User, :restricted_nicknames]) + |> Enum.all?(fn restricted_nickname -> + String.downcase(value) != String.downcase(restricted_nickname) + end) + + if valid?, do: [], else: [nickname: "Invalid nickname"] + end) + end + + def validate_email_not_in_blacklisted_domain(changeset, field) do + validate_change(changeset, field, fn _, value -> + valid? = + Config.get([User, :email_blacklist]) + |> Enum.all?(fn blacklisted_domain -> + blacklisted_domain_downcase = String.downcase(blacklisted_domain) + + !String.ends_with?(String.downcase(value), [ + "@" <> blacklisted_domain_downcase, + "." <> blacklisted_domain_downcase + ]) + end) + + if valid?, do: [], else: [email: "Invalid email"] + end) + end + def maybe_validate_required_email(changeset, true), do: changeset def maybe_validate_required_email(changeset, _) do diff --git a/lib/pleroma/web/activity_pub/activity_pub_controller.ex b/lib/pleroma/web/activity_pub/activity_pub_controller.ex index 20f8bbc2d..b8f63d69d 100644 --- a/lib/pleroma/web/activity_pub/activity_pub_controller.ex +++ b/lib/pleroma/web/activity_pub/activity_pub_controller.ex @@ -84,6 +84,7 @@ defmodule Pleroma.Web.ActivityPub.ActivityPubController do user <- Map.get(assigns, :user, nil), {_, true} <- {:visible?, Visibility.visible_for_user?(object, user)} do conn + |> maybe_skip_cache(user) |> assign(:tracking_fun_data, object.id) |> set_cache_ttl_for(object) |> put_resp_content_type("application/activity+json") @@ -112,6 +113,7 @@ defmodule Pleroma.Web.ActivityPub.ActivityPubController do user <- Map.get(assigns, :user, nil), {_, true} <- {:visible?, Visibility.visible_for_user?(activity, user)} do conn + |> maybe_skip_cache(user) |> maybe_set_tracking_data(activity) |> set_cache_ttl_for(activity) |> put_resp_content_type("application/activity+json") @@ -151,6 +153,15 @@ defmodule Pleroma.Web.ActivityPub.ActivityPubController do assign(conn, :cache_ttl, ttl) end + def maybe_skip_cache(conn, user) do + if user do + conn + |> assign(:skip_cache, true) + else + conn + end + end + # GET /relay/following def relay_following(conn, _params) do with %{halted: false} = conn <- FederatingPlug.call(conn, []) do diff --git a/lib/pleroma/web/activity_pub/mrf/anti_followbot_policy.ex b/lib/pleroma/web/activity_pub/mrf/anti_followbot_policy.ex index 605ca9971..97d75ecf2 100644 --- a/lib/pleroma/web/activity_pub/mrf/anti_followbot_policy.ex +++ b/lib/pleroma/web/activity_pub/mrf/anti_followbot_policy.ex @@ -24,7 +24,7 @@ defmodule Pleroma.Web.ActivityPub.MRF.AntiFollowbotPolicy do defp score_displayname("fedibot"), do: 1.0 defp score_displayname(_), do: 0.0 - defp determine_if_followbot(%User{nickname: nickname, name: displayname}) do + defp determine_if_followbot(%User{nickname: nickname, name: displayname, actor_type: actor_type}) do # nickname will be a binary string except when following a relay nick_score = if is_binary(nickname) do @@ -45,19 +45,32 @@ defmodule Pleroma.Web.ActivityPub.MRF.AntiFollowbotPolicy do 0.0 end - nick_score + name_score + # actor_type "Service" is a Bot account + actor_type_score = + if actor_type == "Service" do + 1.0 + else + 0.0 + end + + nick_score + name_score + actor_type_score end defp determine_if_followbot(_), do: 0.0 + defp bot_allowed?(%{"object" => target}, bot_actor) do + %User{} = user = normalize_by_ap_id(target) + + User.following?(user, bot_actor) + end + @impl true def filter(%{"type" => "Follow", "actor" => actor_id} = message) do %User{} = actor = normalize_by_ap_id(actor_id) score = determine_if_followbot(actor) - # TODO: scan biography data for keywords and score it somehow. - if score < 0.8 do + if score < 0.8 || bot_allowed?(message, actor) do {:ok, message} else {:reject, "[AntiFollowbotPolicy] Scored #{actor_id} as #{score}"} diff --git a/lib/pleroma/web/activity_pub/mrf/steal_emoji_policy.ex b/lib/pleroma/web/activity_pub/mrf/steal_emoji_policy.ex index 06305235e..f66c379b5 100644 --- a/lib/pleroma/web/activity_pub/mrf/steal_emoji_policy.ex +++ b/lib/pleroma/web/activity_pub/mrf/steal_emoji_policy.ex @@ -12,6 +12,14 @@ defmodule Pleroma.Web.ActivityPub.MRF.StealEmojiPolicy do defp accept_host?(host), do: host in Config.get([:mrf_steal_emoji, :hosts], []) + defp shortcode_matches?(shortcode, pattern) when is_binary(pattern) do + shortcode == pattern + end + + defp shortcode_matches?(shortcode, pattern) do + String.match?(shortcode, pattern) + end + defp steal_emoji({shortcode, url}, emoji_dir_path) do url = Pleroma.Web.MediaProxy.url(url) @@ -72,7 +80,7 @@ defmodule Pleroma.Web.ActivityPub.MRF.StealEmojiPolicy do reject_emoji? = [:mrf_steal_emoji, :rejected_shortcodes] |> Config.get([]) - |> Enum.find(false, fn regex -> String.match?(shortcode, regex) end) + |> Enum.find(false, fn pattern -> shortcode_matches?(shortcode, pattern) end) !reject_emoji? end) @@ -122,8 +130,12 @@ defmodule Pleroma.Web.ActivityPub.MRF.StealEmojiPolicy do %{ key: :rejected_shortcodes, type: {:list, :string}, - description: "Regex-list of shortcodes to reject", - suggestions: [""] + description: """ + A list of patterns or matches to reject shortcodes with. + + Each pattern can be a string or [Regex](https://hexdocs.pm/elixir/Regex.html) in the format of `~r/PATTERN/`. + """, + suggestions: ["foo", ~r/foo/] }, %{ key: :size_limit, diff --git a/lib/pleroma/web/activity_pub/object_validators/attachment_validator.ex b/lib/pleroma/web/activity_pub/object_validators/attachment_validator.ex index d1c61ac82..8b641d88d 100644 --- a/lib/pleroma/web/activity_pub/object_validators/attachment_validator.ex +++ b/lib/pleroma/web/activity_pub/object_validators/attachment_validator.ex @@ -12,14 +12,14 @@ defmodule Pleroma.Web.ActivityPub.ObjectValidators.AttachmentValidator do @primary_key false embedded_schema do field(:type, :string) - field(:mediaType, :string, default: "application/octet-stream") + field(:mediaType, ObjectValidators.MIME, default: "application/octet-stream") field(:name, :string) field(:blurhash, :string) embeds_many :url, UrlObjectValidator, primary_key: false do field(:type, :string) field(:href, ObjectValidators.Uri) - field(:mediaType, :string, default: "application/octet-stream") + field(:mediaType, ObjectValidators.MIME, default: "application/octet-stream") field(:width, :integer) field(:height, :integer) end @@ -59,13 +59,7 @@ defmodule Pleroma.Web.ActivityPub.ObjectValidators.AttachmentValidator do end def fix_media_type(data) do - data = Map.put_new(data, "mediaType", data["mimeType"]) - - if is_bitstring(data["mediaType"]) && MIME.extensions(data["mediaType"]) != [] do - data - else - Map.put(data, "mediaType", "application/octet-stream") - end + Map.put_new(data, "mediaType", data["mimeType"]) end defp handle_href(href, mediaType, data) do diff --git a/lib/pleroma/web/activity_pub/transmogrifier.ex b/lib/pleroma/web/activity_pub/transmogrifier.ex index a70330f0e..d6622df86 100644 --- a/lib/pleroma/web/activity_pub/transmogrifier.ex +++ b/lib/pleroma/web/activity_pub/transmogrifier.ex @@ -203,13 +203,13 @@ defmodule Pleroma.Web.ActivityPub.Transmogrifier do media_type = cond do - is_map(url) && MIME.extensions(url["mediaType"]) != [] -> + is_map(url) && url =~ Pleroma.Constants.mime_regex() -> url["mediaType"] - is_bitstring(data["mediaType"]) && MIME.extensions(data["mediaType"]) != [] -> + is_bitstring(data["mediaType"]) && data["mediaType"] =~ Pleroma.Constants.mime_regex() -> data["mediaType"] - is_bitstring(data["mimeType"]) && MIME.extensions(data["mimeType"]) != [] -> + is_bitstring(data["mimeType"]) && data["mimeType"] =~ Pleroma.Constants.mime_regex() -> data["mimeType"] true -> diff --git a/lib/pleroma/web/admin_api/controllers/announcement_controller.ex b/lib/pleroma/web/admin_api/controllers/announcement_controller.ex new file mode 100644 index 000000000..6ad5fc12c --- /dev/null +++ b/lib/pleroma/web/admin_api/controllers/announcement_controller.ex @@ -0,0 +1,83 @@ +# Pleroma: A lightweight social networking server +# Copyright © 2017-2022 Pleroma Authors <https://pleroma.social/> +# SPDX-License-Identifier: AGPL-3.0-only + +defmodule Pleroma.Web.AdminAPI.AnnouncementController do + use Pleroma.Web, :controller + + alias Pleroma.Announcement + alias Pleroma.Web.ControllerHelper + alias Pleroma.Web.Plugs.OAuthScopesPlug + + plug(Pleroma.Web.ApiSpec.CastAndValidate) + plug(OAuthScopesPlug, %{scopes: ["admin:write"]} when action in [:create, :delete, :change]) + plug(OAuthScopesPlug, %{scopes: ["admin:read"]} when action in [:index, :show]) + action_fallback(Pleroma.Web.AdminAPI.FallbackController) + + defdelegate open_api_operation(action), to: Pleroma.Web.ApiSpec.Admin.AnnouncementOperation + + defp default_limit, do: 20 + + def index(conn, params) do + limit = Map.get(params, :limit, default_limit()) + offset = Map.get(params, :offset, 0) + + announcements = Announcement.list_paginated(%{limit: limit, offset: offset}) + + render(conn, "index.json", announcements: announcements) + end + + def show(conn, %{id: id} = _params) do + announcement = Announcement.get_by_id(id) + + if is_nil(announcement) do + {:error, :not_found} + else + render(conn, "show.json", announcement: announcement) + end + end + + def create(%{body_params: params} = conn, _params) do + with {:ok, announcement} <- Announcement.add(change_params(params)) do + render(conn, "show.json", announcement: announcement) + else + _ -> + {:error, 400} + end + end + + def change_params(orig_params) do + data = + %{} + |> Pleroma.Maps.put_if_present("content", orig_params, &Map.fetch(&1, :content)) + |> Pleroma.Maps.put_if_present("all_day", orig_params, &Map.fetch(&1, :all_day)) + + orig_params + |> Map.merge(%{data: data}) + end + + def change(%{body_params: params} = conn, %{id: id} = _params) do + with announcement <- Announcement.get_by_id(id), + {:exists, true} <- {:exists, not is_nil(announcement)}, + {:ok, announcement} <- Announcement.update(announcement, change_params(params)) do + render(conn, "show.json", announcement: announcement) + else + {:exists, false} -> + {:error, :not_found} + + _ -> + {:error, 400} + end + end + + def delete(conn, %{id: id} = _params) do + case Announcement.delete_by_id(id) do + :ok -> + conn + |> ControllerHelper.json_response(:ok, %{}) + + _ -> + {:error, :not_found} + end + end +end diff --git a/lib/pleroma/web/admin_api/views/announcement_view.ex b/lib/pleroma/web/admin_api/views/announcement_view.ex new file mode 100644 index 000000000..a35bd60cf --- /dev/null +++ b/lib/pleroma/web/admin_api/views/announcement_view.ex @@ -0,0 +1,15 @@ +# Pleroma: A lightweight social networking server +# Copyright © 2017-2022 Pleroma Authors <https://pleroma.social/> +# SPDX-License-Identifier: AGPL-3.0-only + +defmodule Pleroma.Web.AdminAPI.AnnouncementView do + use Pleroma.Web, :view + + def render("index.json", %{announcements: announcements}) do + render_many(announcements, __MODULE__, "show.json") + end + + def render("show.json", %{announcement: announcement}) do + Pleroma.Announcement.render_json(announcement, admin: true) + end +end diff --git a/lib/pleroma/web/api_spec/operations/admin/announcement_operation.ex b/lib/pleroma/web/api_spec/operations/admin/announcement_operation.ex new file mode 100644 index 000000000..58a039e72 --- /dev/null +++ b/lib/pleroma/web/api_spec/operations/admin/announcement_operation.ex @@ -0,0 +1,165 @@ +# Pleroma: A lightweight social networking server +# Copyright © 2017-2022 Pleroma Authors <https://pleroma.social/> +# SPDX-License-Identifier: AGPL-3.0-only + +defmodule Pleroma.Web.ApiSpec.Admin.AnnouncementOperation do + alias OpenApiSpex.Operation + alias OpenApiSpex.Schema + alias Pleroma.Web.ApiSpec.Schemas.Announcement + alias Pleroma.Web.ApiSpec.Schemas.ApiError + + import Pleroma.Web.ApiSpec.Helpers + + def open_api_operation(action) do + operation = String.to_existing_atom("#{action}_operation") + apply(__MODULE__, operation, []) + end + + def index_operation do + %Operation{ + tags: ["Announcement managment"], + summary: "Retrieve a list of announcements", + operationId: "AdminAPI.AnnouncementController.index", + security: [%{"oAuth" => ["admin:read"]}], + parameters: [ + Operation.parameter( + :limit, + :query, + %Schema{type: :integer, minimum: 1}, + "the maximum number of announcements to return" + ), + Operation.parameter( + :offset, + :query, + %Schema{type: :integer, minimum: 0}, + "the offset of the first announcement to return" + ) + | admin_api_params() + ], + responses: %{ + 200 => Operation.response("Response", "application/json", list_of_announcements()), + 400 => Operation.response("Forbidden", "application/json", ApiError), + 403 => Operation.response("Forbidden", "application/json", ApiError) + } + } + end + + def show_operation do + %Operation{ + tags: ["Announcement managment"], + summary: "Display one announcement", + operationId: "AdminAPI.AnnouncementController.show", + security: [%{"oAuth" => ["admin:read"]}], + parameters: [ + Operation.parameter( + :id, + :path, + :string, + "announcement id" + ) + | admin_api_params() + ], + responses: %{ + 200 => Operation.response("Response", "application/json", Announcement), + 403 => Operation.response("Forbidden", "application/json", ApiError), + 404 => Operation.response("Not Found", "application/json", ApiError) + } + } + end + + def delete_operation do + %Operation{ + tags: ["Announcement managment"], + summary: "Delete one announcement", + operationId: "AdminAPI.AnnouncementController.delete", + security: [%{"oAuth" => ["admin:write"]}], + parameters: [ + Operation.parameter( + :id, + :path, + :string, + "announcement id" + ) + | admin_api_params() + ], + responses: %{ + 200 => Operation.response("Response", "application/json", %Schema{type: :object}), + 403 => Operation.response("Forbidden", "application/json", ApiError), + 404 => Operation.response("Not Found", "application/json", ApiError) + } + } + end + + def create_operation do + %Operation{ + tags: ["Announcement managment"], + summary: "Create one announcement", + operationId: "AdminAPI.AnnouncementController.create", + security: [%{"oAuth" => ["admin:write"]}], + requestBody: request_body("Parameters", create_request(), required: true), + responses: %{ + 200 => Operation.response("Response", "application/json", Announcement), + 400 => Operation.response("Bad Request", "application/json", ApiError), + 403 => Operation.response("Forbidden", "application/json", ApiError) + } + } + end + + def change_operation do + %Operation{ + tags: ["Announcement managment"], + summary: "Change one announcement", + operationId: "AdminAPI.AnnouncementController.change", + security: [%{"oAuth" => ["admin:write"]}], + parameters: [ + Operation.parameter( + :id, + :path, + :string, + "announcement id" + ) + | admin_api_params() + ], + requestBody: request_body("Parameters", change_request(), required: true), + responses: %{ + 200 => Operation.response("Response", "application/json", Announcement), + 400 => Operation.response("Bad Request", "application/json", ApiError), + 403 => Operation.response("Forbidden", "application/json", ApiError), + 404 => Operation.response("Not Found", "application/json", ApiError) + } + } + end + + defp create_or_change_props do + %{ + content: %Schema{type: :string}, + starts_at: %Schema{type: :string, format: "date-time", nullable: true}, + ends_at: %Schema{type: :string, format: "date-time", nullable: true}, + all_day: %Schema{type: :boolean} + } + end + + def create_request do + %Schema{ + title: "AnnouncementCreateRequest", + type: :object, + required: [:content], + properties: create_or_change_props() + } + end + + def change_request do + %Schema{ + title: "AnnouncementChangeRequest", + type: :object, + properties: create_or_change_props() + } + end + + def list_of_announcements do + %Schema{ + type: :array, + items: Announcement + } + end +end diff --git a/lib/pleroma/web/api_spec/operations/announcement_operation.ex b/lib/pleroma/web/api_spec/operations/announcement_operation.ex new file mode 100644 index 000000000..71be0002a --- /dev/null +++ b/lib/pleroma/web/api_spec/operations/announcement_operation.ex @@ -0,0 +1,57 @@ +# Pleroma: A lightweight social networking server +# Copyright © 2017-2022 Pleroma Authors <https://pleroma.social/> +# SPDX-License-Identifier: AGPL-3.0-only + +defmodule Pleroma.Web.ApiSpec.AnnouncementOperation do + alias OpenApiSpex.Operation + alias OpenApiSpex.Schema + alias Pleroma.Web.ApiSpec.Schemas.Announcement + alias Pleroma.Web.ApiSpec.Schemas.ApiError + + def open_api_operation(action) do + operation = String.to_existing_atom("#{action}_operation") + apply(__MODULE__, operation, []) + end + + def index_operation do + %Operation{ + tags: ["Announcement"], + summary: "Retrieve a list of announcements", + operationId: "MastodonAPI.AnnouncementController.index", + security: [%{"oAuth" => []}], + responses: %{ + 200 => Operation.response("Response", "application/json", list_of_announcements()), + 403 => Operation.response("Forbidden", "application/json", ApiError) + } + } + end + + def mark_read_operation do + %Operation{ + tags: ["Announcement"], + summary: "Mark one announcement as read", + operationId: "MastodonAPI.AnnouncementController.mark_read", + security: [%{"oAuth" => ["write:accounts"]}], + parameters: [ + Operation.parameter( + :id, + :path, + :string, + "announcement id" + ) + ], + responses: %{ + 200 => Operation.response("Response", "application/json", %Schema{type: :object}), + 403 => Operation.response("Forbidden", "application/json", ApiError), + 404 => Operation.response("Not Found", "application/json", ApiError) + } + } + end + + def list_of_announcements do + %Schema{ + type: :array, + items: Announcement + } + end +end diff --git a/lib/pleroma/web/api_spec/schemas/announcement.ex b/lib/pleroma/web/api_spec/schemas/announcement.ex new file mode 100644 index 000000000..67d129ef6 --- /dev/null +++ b/lib/pleroma/web/api_spec/schemas/announcement.ex @@ -0,0 +1,45 @@ +# Pleroma: A lightweight social networking server +# Copyright © 2017-2022 Pleroma Authors <https://pleroma.social/> +# SPDX-License-Identifier: AGPL-3.0-only + +defmodule Pleroma.Web.ApiSpec.Schemas.Announcement do + alias OpenApiSpex.Schema + alias Pleroma.Web.ApiSpec.Schemas.FlakeID + + require OpenApiSpex + + OpenApiSpex.schema(%{ + title: "Announcement", + description: "Response schema for an announcement", + type: :object, + properties: %{ + id: FlakeID, + content: %Schema{type: :string}, + starts_at: %Schema{ + type: :string, + format: "date-time", + nullable: true + }, + ends_at: %Schema{ + type: :string, + format: "date-time", + nullable: true + }, + all_day: %Schema{type: :boolean}, + published_at: %Schema{type: :string, format: "date-time"}, + updated_at: %Schema{type: :string, format: "date-time"}, + read: %Schema{type: :boolean}, + mentions: %Schema{type: :array}, + statuses: %Schema{type: :array}, + tags: %Schema{type: :array}, + emojis: %Schema{type: :array}, + reactions: %Schema{type: :array}, + pleroma: %Schema{ + type: :object, + properties: %{ + raw_content: %Schema{type: :string} + } + } + } + }) +end diff --git a/lib/pleroma/web/gettext.ex b/lib/pleroma/web/gettext.ex index d5ec13b66..5ef49d841 100644 --- a/lib/pleroma/web/gettext.ex +++ b/lib/pleroma/web/gettext.ex @@ -161,10 +161,25 @@ defmodule Pleroma.Web.Gettext do end end + # We do not yet have a proper English translation. The "English" + # version is currently but the fallback msgid. However, this + # will not work if the user puts English as the first language, + # and at the same time specifies other languages, as gettext will + # think the English translation is missing, and call + # handle_missing_translation functions. This may result in + # text in other languages being shown even if English is preferred + # by the user. + # + # To prevent this, we do not allow fallbacking when the current + # locale missing a translation is English. + defp should_fallback?(locale) do + locale != "en" + end + def handle_missing_translation(locale, domain, msgctxt, msgid, bindings) do next = next_locale(locale, get_locales()) - if is_nil(next) do + if is_nil(next) or not should_fallback?(locale) do super(locale, domain, msgctxt, msgid, bindings) else {:ok, @@ -185,7 +200,7 @@ defmodule Pleroma.Web.Gettext do ) do next = next_locale(locale, get_locales()) - if is_nil(next) do + if is_nil(next) or not should_fallback?(locale) do super(locale, domain, msgctxt, msgid, msgid_plural, n, bindings) else {:ok, diff --git a/lib/pleroma/web/mastodon_api/controllers/announcement_controller.ex b/lib/pleroma/web/mastodon_api/controllers/announcement_controller.ex new file mode 100644 index 000000000..080af96d5 --- /dev/null +++ b/lib/pleroma/web/mastodon_api/controllers/announcement_controller.ex @@ -0,0 +1,60 @@ +# Pleroma: A lightweight social networking server +# Copyright © 2017-2022 Pleroma Authors <https://pleroma.social/> +# SPDX-License-Identifier: AGPL-3.0-only + +defmodule Pleroma.Web.MastodonAPI.AnnouncementController do + use Pleroma.Web, :controller + + import Pleroma.Web.ControllerHelper, + only: [ + json_response: 3 + ] + + alias Pleroma.Announcement + alias Pleroma.Web.Plugs.OAuthScopesPlug + + plug(Pleroma.Web.ApiSpec.CastAndValidate) + + # Mastodon docs say this only requires a user token, no scopes needed + # As the op `|` requires at least one scope to be present, we use `&` here. + plug( + OAuthScopesPlug, + %{scopes: [], op: :&} + when action in [:index] + ) + + # Same as in MastodonAPI specs + plug( + OAuthScopesPlug, + %{scopes: ["write:accounts"]} + when action in [:mark_read] + ) + + action_fallback(Pleroma.Web.MastodonAPI.FallbackController) + + defdelegate open_api_operation(action), to: Pleroma.Web.ApiSpec.AnnouncementOperation + + @doc "GET /api/v1/announcements" + def index(%{assigns: %{user: user}} = conn, _params) do + render(conn, "index.json", announcements: all_visible(), user: user) + end + + def index(conn, _params) do + render(conn, "index.json", announcements: all_visible(), user: nil) + end + + defp all_visible do + Announcement.list_all_visible() + end + + @doc "POST /api/v1/announcements/:id/dismiss" + def mark_read(%{assigns: %{user: user}} = conn, %{id: id} = _params) do + with announcement when not is_nil(announcement) <- Announcement.get_by_id(id), + {:ok, _} <- Announcement.mark_read_by(announcement, user) do + json_response(conn, :ok, %{}) + else + _ -> + {:error, :not_found} + end + end +end diff --git a/lib/pleroma/web/mastodon_api/views/announcement_view.ex b/lib/pleroma/web/mastodon_api/views/announcement_view.ex new file mode 100644 index 000000000..93fdfb1f1 --- /dev/null +++ b/lib/pleroma/web/mastodon_api/views/announcement_view.ex @@ -0,0 +1,15 @@ +# Pleroma: A lightweight social networking server +# Copyright © 2017-2022 Pleroma Authors <https://pleroma.social/> +# SPDX-License-Identifier: AGPL-3.0-only + +defmodule Pleroma.Web.MastodonAPI.AnnouncementView do + use Pleroma.Web, :view + + def render("index.json", %{announcements: announcements, user: user}) do + render_many(announcements, __MODULE__, "show.json", user: user) + end + + def render("show.json", %{announcement: announcement, user: user}) do + Pleroma.Announcement.render_json(announcement, for: user) + end +end diff --git a/lib/pleroma/web/plugs/cache.ex b/lib/pleroma/web/plugs/cache.ex index e2cf5759d..667477857 100644 --- a/lib/pleroma/web/plugs/cache.ex +++ b/lib/pleroma/web/plugs/cache.ex @@ -97,13 +97,21 @@ defmodule Pleroma.Web.Plugs.Cache do key = cache_key(conn, opts) content_type = content_type(conn) + should_cache = not Map.get(conn.assigns, :skip_cache, false) + conn = unless opts[:tracking_fun] do - @cachex.put(:web_resp_cache, key, {content_type, body}, ttl: ttl) + if should_cache do + @cachex.put(:web_resp_cache, key, {content_type, body}, ttl: ttl) + end + conn else tracking_fun_data = Map.get(conn.assigns, :tracking_fun_data, nil) - @cachex.put(:web_resp_cache, key, {content_type, body, tracking_fun_data}, ttl: ttl) + + if should_cache do + @cachex.put(:web_resp_cache, key, {content_type, body, tracking_fun_data}, ttl: ttl) + end opts.tracking_fun.(conn, tracking_fun_data) end diff --git a/lib/pleroma/web/router.ex b/lib/pleroma/web/router.ex index ceb6c3cfd..7bbc20275 100644 --- a/lib/pleroma/web/router.ex +++ b/lib/pleroma/web/router.ex @@ -229,6 +229,12 @@ defmodule Pleroma.Web.Router do post("/frontends/install", FrontendController, :install) post("/backups", AdminAPIController, :create_backup) + + get("/announcements", AnnouncementController, :index) + post("/announcements", AnnouncementController, :create) + get("/announcements/:id", AnnouncementController, :show) + patch("/announcements/:id", AnnouncementController, :change) + delete("/announcements/:id", AnnouncementController, :delete) end # AdminAPI: admins and mods (staff) can perform these actions (if enabled by config) @@ -575,6 +581,9 @@ defmodule Pleroma.Web.Router do get("/timelines/home", TimelineController, :home) get("/timelines/direct", TimelineController, :direct) get("/timelines/list/:list_id", TimelineController, :list) + + get("/announcements", AnnouncementController, :index) + post("/announcements/:id/dismiss", AnnouncementController, :mark_read) end scope "/api/v1", Pleroma.Web.MastodonAPI do diff --git a/lib/pleroma/web/twitter_api/views/remote_follow_view.ex b/lib/pleroma/web/twitter_api/views/remote_follow_view.ex index bd33d4c0a..8902261b0 100644 --- a/lib/pleroma/web/twitter_api/views/remote_follow_view.ex +++ b/lib/pleroma/web/twitter_api/views/remote_follow_view.ex @@ -7,5 +7,9 @@ defmodule Pleroma.Web.TwitterAPI.RemoteFollowView do import Phoenix.HTML.Form alias Pleroma.Web.Gettext - defdelegate avatar_url(user), to: Pleroma.User + def avatar_url(user) do + user + |> Pleroma.User.avatar_url() + |> Pleroma.Web.MediaProxy.url() + end end @@ -4,7 +4,7 @@ defmodule Pleroma.Mixfile do def project do [ app: :pleroma, - version: version("2.4.51"), + version: version("2.4.52"), elixir: "~> 1.9", elixirc_paths: elixirc_paths(Mix.env()), compilers: [:phoenix, :gettext] ++ Mix.compilers(), @@ -145,7 +145,7 @@ defmodule Pleroma.Mixfile do {:mogrify, "~> 0.9.1"}, {:ex_aws, "~> 2.1.6"}, {:ex_aws_s3, "~> 2.0"}, - {:sweet_xml, "~> 0.6.6"}, + {:sweet_xml, "~> 0.7.2"}, {:earmark, "~> 1.4.15"}, {:bbcode_pleroma, "~> 0.2.0"}, {:crypt, @@ -194,7 +194,7 @@ defmodule Pleroma.Mixfile do ref: "e0f16822d578866e186a0974d65ad58cddc1e2ab"}, {:restarter, path: "./restarter"}, {:majic, "~> 1.0"}, - {:eblurhash, "~> 1.1.0"}, + {:eblurhash, "~> 1.2.2"}, {:open_api_spex, "~> 3.10"}, {:phoenix_live_dashboard, "~> 0.6.2"}, {:ecto_psql_extras, "~> 0.6"}, @@ -29,7 +29,7 @@ "deep_merge": {:hex, :deep_merge, "1.0.0", "b4aa1a0d1acac393bdf38b2291af38cb1d4a52806cf7a4906f718e1feb5ee961", [:mix], [], "hexpm", "ce708e5f094b9cd4e8f2be4f00d2f4250c4095be93f8cd6d018c753894885430"}, "earmark": {:hex, :earmark, "1.4.18", "618c4ff1563450d1832b7fb41dc6755e470f91a6fd4c70f350a58b14f64a7db8", [:mix], [{:earmark_parser, ">= 1.4.17", [hex: :earmark_parser, repo: "hexpm", optional: false]}], "hexpm", "57ac3b6da3958ed09c669a9b159e86377fcccda56bacde8a209fa4dcdef52560"}, "earmark_parser": {:hex, :earmark_parser, "1.4.17", "6f3c7e94170377ba45241d394389e800fb15adc5de51d0a3cd52ae766aafd63f", [:mix], [], "hexpm", "f93ac89c9feca61c165b264b5837bf82344d13bebc634cd575cb711e2e342023"}, - "eblurhash": {:hex, :eblurhash, "1.1.0", "e10ccae762598507ebfacf0b645ed49520f2afa3e7e9943e73a91117dffce415", [:rebar3], [], "hexpm", "2e6b889d09fddd374e3c5ac57c486138768763264e99ac1074ae5fa7fc9ab51d"}, + "eblurhash": {:hex, :eblurhash, "1.2.2", "7da4255aaea984b31bb71155f673257353b0e0554d0d30dcf859547e74602582", [:rebar3], [], "hexpm", "8c20ca00904de023a835a9dcb7b7762fed32264c85a80c3cafa85288e405044c"}, "ecto": {:hex, :ecto, "3.6.2", "efdf52acfc4ce29249bab5417415bd50abd62db7b0603b8bab0d7b996548c2bc", [:mix], [{:decimal, "~> 1.6 or ~> 2.0", [hex: :decimal, repo: "hexpm", optional: false]}, {:jason, "~> 1.0", [hex: :jason, repo: "hexpm", optional: true]}, {:telemetry, "~> 0.4", [hex: :telemetry, repo: "hexpm", optional: false]}], "hexpm", "efad6dfb04e6f986b8a3047822b0f826d9affe8e4ebdd2aeedbfcb14fd48884e"}, "ecto_enum": {:hex, :ecto_enum, "1.4.0", "d14b00e04b974afc69c251632d1e49594d899067ee2b376277efd8233027aec8", [:mix], [{:ecto, ">= 3.0.0", [hex: :ecto, repo: "hexpm", optional: false]}, {:ecto_sql, "> 3.0.0", [hex: :ecto_sql, repo: "hexpm", optional: false]}, {:mariaex, ">= 0.0.0", [hex: :mariaex, repo: "hexpm", optional: true]}, {:postgrex, ">= 0.0.0", [hex: :postgrex, repo: "hexpm", optional: true]}], "hexpm", "8fb55c087181c2b15eee406519dc22578fa60dd82c088be376d0010172764ee4"}, "ecto_psql_extras": {:hex, :ecto_psql_extras, "0.7.4", "5d43fd088d39a158c860b17e8d210669587f63ec89ea122a4654861c8c6e2db4", [:mix], [{:ecto_sql, "~> 3.4", [hex: :ecto_sql, repo: "hexpm", optional: false]}, {:postgrex, ">= 0.15.7", [hex: :postgrex, repo: "hexpm", optional: false]}, {:table_rex, "~> 3.1.1", [hex: :table_rex, repo: "hexpm", optional: false]}], "hexpm", "311db02f1b772e3d0dc7f56a05044b5e1499d78ed6abf38885e1ca70059449e5"}, @@ -120,7 +120,7 @@ "remote_ip": {:git, "https://git.pleroma.social/pleroma/remote_ip.git", "b647d0deecaa3acb140854fe4bda5b7e1dc6d1c8", [ref: "b647d0deecaa3acb140854fe4bda5b7e1dc6d1c8"]}, "sleeplocks": {:hex, :sleeplocks, "1.1.1", "3d462a0639a6ef36cc75d6038b7393ae537ab394641beb59830a1b8271faeed3", [:rebar3], [], "hexpm", "84ee37aeff4d0d92b290fff986d6a95ac5eedf9b383fadfd1d88e9b84a1c02e1"}, "ssl_verify_fun": {:hex, :ssl_verify_fun, "1.1.6", "cf344f5692c82d2cd7554f5ec8fd961548d4fd09e7d22f5b62482e5aeaebd4b0", [:make, :mix, :rebar3], [], "hexpm", "bdb0d2471f453c88ff3908e7686f86f9be327d065cc1ec16fa4540197ea04680"}, - "sweet_xml": {:hex, :sweet_xml, "0.6.6", "fc3e91ec5dd7c787b6195757fbcf0abc670cee1e4172687b45183032221b66b8", [:mix], [], "hexpm", "2e1ec458f892ffa81f9f8386e3f35a1af6db7a7a37748a64478f13163a1f3573"}, + "sweet_xml": {:hex, :sweet_xml, "0.7.2", "4729f997286811fabdd8288f8474e0840a76573051062f066c4b597e76f14f9f", [:mix], [], "hexpm", "6894e68a120f454534d99045ea3325f7740ea71260bc315f82e29731d570a6e8"}, "swoosh": {:hex, :swoosh, "1.3.11", "34f79c57f19892b43bd2168de9ff5de478a721a26328ef59567aad4243e7a77b", [:mix], [{:cowboy, "~> 1.1 or ~> 2.4", [hex: :cowboy, repo: "hexpm", optional: true]}, {:finch, "~> 0.6", [hex: :finch, repo: "hexpm", optional: true]}, {:gen_smtp, "~> 0.13 or ~> 1.0", [hex: :gen_smtp, repo: "hexpm", optional: true]}, {:hackney, "~> 1.9", [hex: :hackney, repo: "hexpm", optional: true]}, {:jason, "~> 1.0", [hex: :jason, repo: "hexpm", optional: false]}, {:mail, "~> 0.2", [hex: :mail, repo: "hexpm", optional: true]}, {:mime, "~> 1.1", [hex: :mime, repo: "hexpm", optional: false]}, {:plug_cowboy, ">= 1.0.0", [hex: :plug_cowboy, repo: "hexpm", optional: true]}], "hexpm", "f1e2a048db454f9982b9cf840f75e7399dd48be31ecc2a7dc10012a803b913af"}, "syslog": {:hex, :syslog, "1.1.0", "6419a232bea84f07b56dc575225007ffe34d9fdc91abe6f1b2f254fd71d8efc2", [:rebar3], [], "hexpm", "4c6a41373c7e20587be33ef841d3de6f3beba08519809329ecc4d27b15b659e1"}, "table_rex": {:hex, :table_rex, "3.1.1", "0c67164d1714b5e806d5067c1e96ff098ba7ae79413cc075973e17c38a587caa", [:mix], [], "hexpm", "678a23aba4d670419c23c17790f9dcd635a4a89022040df7d5d772cb21012490"}, diff --git a/priv/repo/migrations/20220220135625_upload_filter_exiftool_to_exiftool_strip_location.exs b/priv/repo/migrations/20220220135625_upload_filter_exiftool_to_exiftool_strip_location.exs new file mode 100644 index 000000000..0878b9699 --- /dev/null +++ b/priv/repo/migrations/20220220135625_upload_filter_exiftool_to_exiftool_strip_location.exs @@ -0,0 +1,37 @@ +defmodule Pleroma.Repo.Migrations.UploadFilterExiftoolToExiftoolStripLocation do + use Ecto.Migration + + alias Pleroma.ConfigDB + + def up, + do: + ConfigDB.get_by_params(%{group: :pleroma, key: Pleroma.Upload}) + |> update_filtername( + Pleroma.Upload.Filter.Exiftool, + Pleroma.Upload.Filter.Exiftool.StripLocation + ) + + def down, + do: + ConfigDB.get_by_params(%{group: :pleroma, key: Pleroma.Upload}) + |> update_filtername( + Pleroma.Upload.Filter.Exiftool.StripLocation, + Pleroma.Upload.Filter.Exiftool + ) + + defp update_filtername(%{value: value}, from_filtername, to_filtername) do + new_value = + value + |> Keyword.update(:filters, [], fn filters -> + filters + |> Enum.map(fn + ^from_filtername -> to_filtername + filter -> filter + end) + end) + + ConfigDB.update_or_create(%{group: :pleroma, key: Pleroma.Upload, value: new_value}) + end + + defp update_filtername(_, _, _), do: nil +end diff --git a/priv/repo/migrations/20220308012601_create_announcements.exs b/priv/repo/migrations/20220308012601_create_announcements.exs new file mode 100644 index 000000000..01c9ce041 --- /dev/null +++ b/priv/repo/migrations/20220308012601_create_announcements.exs @@ -0,0 +1,26 @@ +defmodule Pleroma.Repo.Migrations.CreateAnnouncements do + use Ecto.Migration + + def change do + create_if_not_exists table(:announcements, primary_key: false) do + add(:id, :uuid, primary_key: true) + add(:data, :map) + add(:starts_at, :naive_datetime) + add(:ends_at, :naive_datetime) + add(:rendered, :map) + + timestamps() + end + + create_if_not_exists table(:announcement_read_relationships) do + add(:user_id, references(:users, type: :uuid, on_delete: :delete_all)) + add(:announcement_id, references(:announcements, type: :uuid, on_delete: :delete_all)) + + timestamps(updated_at: false) + end + + create_if_not_exists( + unique_index(:announcement_read_relationships, [:user_id, :announcement_id]) + ) + end +end diff --git a/priv/repo/migrations/20220506175506_add_index_hotspots.exs b/priv/repo/migrations/20220506175506_add_index_hotspots.exs new file mode 100644 index 000000000..e1f59bbac --- /dev/null +++ b/priv/repo/migrations/20220506175506_add_index_hotspots.exs @@ -0,0 +1,17 @@ +# Pleroma: A lightweight social networking server +# Copyright © 2017-2022 Pleroma Authors <https://pleroma.social/> +# SPDX-License-Identifier: AGPL-3.0-only + +defmodule Pleroma.Repo.Migrations.AddIndexHotspots do + use Ecto.Migration + + def change do + # Stop inserts into activities from doing a full-table scan of users: + create_if_not_exists(index(:users, [:ap_id, "COALESCE(follower_address, '')"])) + + # Change two indexes and a filter recheck into one index scan: + create_if_not_exists(index(:following_relationships, [:follower_id, :state])) + + create_if_not_exists(index(:notifications, [:user_id, :seen])) + end +end diff --git a/priv/repo/migrations/20220602052233_change_report_notes_content_to_text.exs b/priv/repo/migrations/20220602052233_change_report_notes_content_to_text.exs new file mode 100644 index 000000000..9343373fc --- /dev/null +++ b/priv/repo/migrations/20220602052233_change_report_notes_content_to_text.exs @@ -0,0 +1,16 @@ +defmodule Pleroma.Repo.Migrations.ChangeReportNotesContentToText do + use Ecto.Migration + + def up do + alter table(:report_notes) do + modify(:content, :text) + end + end + + # 20191203043610_create_report_notes.exs + def down do + alter table(:report_notes) do + modify(:content, :string) + end + end +end diff --git a/test/fixtures/image_with_caption-abstract.jpg b/test/fixtures/image_with_caption-abstract.jpg Binary files differnew file mode 100644 index 000000000..f982ffa81 --- /dev/null +++ b/test/fixtures/image_with_caption-abstract.jpg diff --git a/test/fixtures/image_with_imagedescription_and_caption-abstract.jpg b/test/fixtures/image_with_imagedescription_and_caption-abstract.jpg Binary files differnew file mode 100644 index 000000000..c82a269ef --- /dev/null +++ b/test/fixtures/image_with_imagedescription_and_caption-abstract.jpg diff --git a/test/fixtures/image_with_imagedescription_and_caption-abstract_whitespaces.jpg b/test/fixtures/image_with_imagedescription_and_caption-abstract_whitespaces.jpg Binary files differnew file mode 100644 index 000000000..a232fd2a1 --- /dev/null +++ b/test/fixtures/image_with_imagedescription_and_caption-abstract_whitespaces.jpg diff --git a/test/fixtures/image_with_no_description.jpg b/test/fixtures/image_with_no_description.jpg Binary files differnew file mode 100644 index 000000000..ec6fc4be8 --- /dev/null +++ b/test/fixtures/image_with_no_description.jpg diff --git a/test/mix/tasks/pleroma/instance_test.exs b/test/mix/tasks/pleroma/instance_test.exs index 249689ec6..b1c10e03c 100644 --- a/test/mix/tasks/pleroma/instance_test.exs +++ b/test/mix/tasks/pleroma/instance_test.exs @@ -67,7 +67,9 @@ defmodule Mix.Tasks.Pleroma.InstanceTest do "test/uploads", "--static-dir", "./test/../test/instance/static/", - "--strip-uploads", + "--strip-uploads-location", + "y", + "--read-uploads-description", "y", "--dedupe-uploads", "n", @@ -91,7 +93,10 @@ defmodule Mix.Tasks.Pleroma.InstanceTest do assert generated_config =~ "password: \"dbpass\"" assert generated_config =~ "configurable_from_database: true" assert generated_config =~ "http: [ip: {127, 0, 0, 1}, port: 4000]" - assert generated_config =~ "filters: [Pleroma.Upload.Filter.Exiftool]" + + assert generated_config =~ + "filters: [Pleroma.Upload.Filter.Exiftool.StripLocation, Pleroma.Upload.Filter.Exiftool.ReadDescription]" + assert File.read!(tmp_path() <> "setup.psql") == generated_setup_psql() assert File.exists?(Path.expand("./test/instance/static/robots.txt")) end diff --git a/test/pleroma/announcement_read_relationship_test.exs b/test/pleroma/announcement_read_relationship_test.exs new file mode 100644 index 000000000..5fd4ffbef --- /dev/null +++ b/test/pleroma/announcement_read_relationship_test.exs @@ -0,0 +1,40 @@ +# Pleroma: A lightweight social networking server +# Copyright © 2017-2022 Pleroma Authors <https://pleroma.social/> +# SPDX-License-Identifier: AGPL-3.0-only + +defmodule Pleroma.AnnouncementReadRelationshipTest do + alias Pleroma.AnnouncementReadRelationship + + use Pleroma.DataCase, async: true + + import Pleroma.Factory + + setup do + {:ok, user: insert(:user), announcement: insert(:announcement)} + end + + describe "mark_read/2" do + test "should insert relationship", %{user: user, announcement: announcement} do + {:ok, _} = AnnouncementReadRelationship.mark_read(user, announcement) + + assert AnnouncementReadRelationship.exists?(user, announcement) + end + end + + describe "mark_unread/2" do + test "should delete relationship", %{user: user, announcement: announcement} do + {:ok, _} = AnnouncementReadRelationship.mark_read(user, announcement) + + assert :ok = AnnouncementReadRelationship.mark_unread(user, announcement) + refute AnnouncementReadRelationship.exists?(user, announcement) + end + + test "should not fail if relationship does not exist", %{ + user: user, + announcement: announcement + } do + assert :ok = AnnouncementReadRelationship.mark_unread(user, announcement) + refute AnnouncementReadRelationship.exists?(user, announcement) + end + end +end diff --git a/test/pleroma/announcement_test.exs b/test/pleroma/announcement_test.exs new file mode 100644 index 000000000..a007c3718 --- /dev/null +++ b/test/pleroma/announcement_test.exs @@ -0,0 +1,98 @@ +# Pleroma: A lightweight social networking server +# Copyright © 2017-2022 Pleroma Authors <https://pleroma.social/> +# SPDX-License-Identifier: AGPL-3.0-only + +defmodule Pleroma.AnnouncementTest do + alias Pleroma.Announcement + + use Pleroma.DataCase, async: true + + import Pleroma.Factory + + describe "list_all_visible_when/1" do + setup do: {:ok, time: NaiveDateTime.utc_now()} + + test "with no start or end time", %{time: time} do + _announcement = insert(:announcement) + + assert [_] = Announcement.list_all_visible_when(time) + end + + test "with start time before current", %{time: time} do + before_now = NaiveDateTime.add(time, -10, :second) + + _announcement = insert(:announcement, %{starts_at: before_now}) + + assert [_] = Announcement.list_all_visible_when(time) + end + + test "with start time after current", %{time: time} do + after_now = NaiveDateTime.add(time, 10, :second) + + _announcement = insert(:announcement, %{starts_at: after_now}) + + assert [] = Announcement.list_all_visible_when(time) + end + + test "with end time after current", %{time: time} do + after_now = NaiveDateTime.add(time, 10, :second) + + _announcement = insert(:announcement, %{ends_at: after_now}) + + assert [_] = Announcement.list_all_visible_when(time) + end + + test "with end time before current", %{time: time} do + before_now = NaiveDateTime.add(time, -10, :second) + + _announcement = insert(:announcement, %{ends_at: before_now}) + + assert [] = Announcement.list_all_visible_when(time) + end + + test "with both start and end time", %{time: time} do + before_now = NaiveDateTime.add(time, -10, :second) + after_now = NaiveDateTime.add(time, 10, :second) + + _announcement = insert(:announcement, %{starts_at: before_now, ends_at: after_now}) + + assert [_] = Announcement.list_all_visible_when(time) + end + + test "with both start and end time, current not in the range", %{time: time} do + before_now = NaiveDateTime.add(time, -10, :second) + after_now = NaiveDateTime.add(time, 10, :second) + + _announcement = insert(:announcement, %{starts_at: after_now, ends_at: before_now}) + + assert [] = Announcement.list_all_visible_when(time) + end + end + + describe "announcements formatting" do + test "it formats links" do + raw = "something on https://pleroma.social ." + announcement = insert(:announcement, %{data: %{"content" => raw}}) + + assert announcement.rendered["content"] =~ ~r(<a.+?https://pleroma.social) + assert announcement.data["content"] == raw + end + + test "it formats mentions" do + user = insert(:user) + raw = "something on @#{user.nickname} ." + announcement = insert(:announcement, %{data: %{"content" => raw}}) + + assert announcement.rendered["content"] =~ ~r(<a.+?#{user.nickname}) + assert announcement.data["content"] == raw + end + + test "it formats tags" do + raw = "something on #mew ." + announcement = insert(:announcement, %{data: %{"content" => raw}}) + + assert announcement.rendered["content"] =~ ~r(<a.+?#mew) + assert announcement.data["content"] == raw + end + end +end diff --git a/test/pleroma/config/deprecation_warnings_test.exs b/test/pleroma/config/deprecation_warnings_test.exs index 202ec4b90..f3453ddb0 100644 --- a/test/pleroma/config/deprecation_warnings_test.exs +++ b/test/pleroma/config/deprecation_warnings_test.exs @@ -11,6 +11,62 @@ defmodule Pleroma.Config.DeprecationWarningsTest do alias Pleroma.Config alias Pleroma.Config.DeprecationWarnings + describe "filter exiftool" do + test "gives warning when still used" do + clear_config( + [Pleroma.Upload, :filters], + [Pleroma.Upload.Filter.Exiftool] + ) + + assert capture_log(fn -> DeprecationWarnings.check_exiftool_filter() end) =~ + """ + !!!DEPRECATION WARNING!!! + Your config is using Exiftool as a filter instead of Exiftool.StripLocation. This should work for now, but you are advised to change to the new configuration to prevent possible issues later: + + ``` + config :pleroma, Pleroma.Upload, + filters: [Pleroma.Upload.Filter.Exiftool] + ``` + + Is now + + + ``` + config :pleroma, Pleroma.Upload, + filters: [Pleroma.Upload.Filter.Exiftool.StripLocation] + ``` + """ + end + + test "changes setting to exiftool strip location" do + clear_config( + [Pleroma.Upload, :filters], + [Pleroma.Upload.Filter.Exiftool, Pleroma.Upload.Filter.Exiftool.ReadDescription] + ) + + expected_config = [ + Pleroma.Upload.Filter.Exiftool.StripLocation, + Pleroma.Upload.Filter.Exiftool.ReadDescription + ] + + capture_log(fn -> DeprecationWarnings.warn() end) + + assert Config.get([Pleroma.Upload]) |> Keyword.get(:filters, []) == expected_config + end + + test "doesn't give a warning with correct config" do + clear_config( + [Pleroma.Upload, :filters], + [ + Pleroma.Upload.Filter.Exiftool.StripLocation, + Pleroma.Upload.Filter.Exiftool.ReadDescription + ] + ) + + assert capture_log(fn -> DeprecationWarnings.check_exiftool_filter() end) == "" + end + end + describe "simple policy tuples" do test "gives warning when there are still strings" do clear_config([:mrf_simple], diff --git a/test/pleroma/config_db_test.exs b/test/pleroma/config_db_test.exs index ea44b716f..ba7c615e2 100644 --- a/test/pleroma/config_db_test.exs +++ b/test/pleroma/config_db_test.exs @@ -238,10 +238,11 @@ defmodule Pleroma.ConfigDBTest do end test "ssl options" do - assert ConfigDB.to_elixir_types([":tlsv1", ":tlsv1.1", ":tlsv1.2"]) == [ + assert ConfigDB.to_elixir_types([":tlsv1", ":tlsv1.1", ":tlsv1.2", ":tlsv1.3"]) == [ :tlsv1, :"tlsv1.1", - :"tlsv1.2" + :"tlsv1.2", + :"tlsv1.3" ] end diff --git a/test/pleroma/docs/generator_test.exs b/test/pleroma/docs/generator_test.exs index 92b4d13e4..5e41d6ccb 100644 --- a/test/pleroma/docs/generator_test.exs +++ b/test/pleroma/docs/generator_test.exs @@ -86,7 +86,7 @@ defmodule Pleroma.Docs.GeneratorTest do key: :versions, type: {:list, :atom}, description: "List of TLS version to use", - suggestions: [:tlsv1, ":tlsv1.1", ":tlsv1.2"] + suggestions: [:tlsv1, ":tlsv1.1", ":tlsv1.2", ":tlsv1.3"] } ] }, @@ -213,7 +213,7 @@ defmodule Pleroma.Docs.GeneratorTest do test "suggestion for tls versions" do [%{children: children} | _] = Generator.convert_to_strings(@descriptions) child = Enum.at(children, 8) - assert child[:suggestions] == [":tlsv1", ":tlsv1.1", ":tlsv1.2"] + assert child[:suggestions] == [":tlsv1", ":tlsv1.1", ":tlsv1.2", ":tlsv1.3"] end test "subgroup with module name" do diff --git a/test/pleroma/report_note_test.exs b/test/pleroma/report_note_test.exs index 136ad9a40..e2028a3ba 100644 --- a/test/pleroma/report_note_test.exs +++ b/test/pleroma/report_note_test.exs @@ -13,4 +13,27 @@ defmodule Pleroma.ReportNoteTest do assert {:ok, note} = ReportNote.create(user.id, report.id, "naughty boy") assert note.content == "naughty boy" end + + test "create/3 with very long content" do + user = insert(:user) + report = insert(:report_activity) + + very_long_content = """ + ] pwgen 25 15 + eJ9eeceiquoolei2queeLeimi aiN9ie2iokie8chush7aiph5N ulaNgaighoPiequaipuzoog8F + Ohphei0hee6hoo0wah4Aasah9 ziel3Yo3eew4neiy3ekiesh8u ue9ShahTh7oongoPheeneijah + ohGheeCh6aloque0Neviopou3 ush2oobohxeec4aequeich3Oo Ze3eighoowiojadohch8iCa1n + Yu4yieBie9eengoich8fae4th chohqu6exooSiibogh3iefeez peephahtaik9quie5mohD9nee + eeQuur3rie5mei8ieng6iesie wei1meinguv0Heidoov8Ibaed deemo2Poh6ohc3eiBeez1uox2 + ] pwgen 25 15 + eJ9eeceiquoolei2queeLeimi aiN9ie2iokie8chush7aiph5N ulaNgaighoPiequaipuzoog8F + Ohphei0hee6hoo0wah4Aasah9 ziel3Yo3eew4neiy3ekiesh8u ue9ShahTh7oongoPheeneijah + ohGheeCh6aloque0Neviopou3 ush2oobohxeec4aequeich3Oo Ze3eighoowiojadohch8iCa1n + Yu4yieBie9eengoich8fae4th chohqu6exooSiibogh3iefeez peephahtaik9quie5mohD9nee + eeQuur3rie5mei8ieng6iesie wei1meinguv0Heidoov8Ibaed deemo2Poh6ohc3eiBeez1uox2 + """ + + assert {:ok, note} = ReportNote.create(user.id, report.id, very_long_content) + assert note.content == very_long_content + end end diff --git a/test/pleroma/upload/filter/exiftool/read_description_test.exs b/test/pleroma/upload/filter/exiftool/read_description_test.exs new file mode 100644 index 000000000..7389fda47 --- /dev/null +++ b/test/pleroma/upload/filter/exiftool/read_description_test.exs @@ -0,0 +1,117 @@ +# Pleroma: A lightweight social networking server +# Copyright © 2017-2021 Pleroma Authors <https://pleroma.social/> +# SPDX-License-Identifier: AGPL-3.0-only + +defmodule Pleroma.Upload.Filter.Exiftool.ReadDescriptionTest do + use Pleroma.DataCase, async: true + alias Pleroma.Upload.Filter + + @uploads %Pleroma.Upload{ + name: "image_with_imagedescription_and_caption-abstract.jpg", + content_type: "image/jpeg", + path: Path.absname("test/fixtures/image_with_imagedescription_and_caption-abstract.jpg"), + tempfile: Path.absname("test/fixtures/image_with_imagedescription_and_caption-abstract.jpg"), + description: nil + } + + test "keeps description when not empty" do + uploads = %Pleroma.Upload{ + name: "image_with_imagedescription_and_caption-abstract.jpg", + content_type: "image/jpeg", + path: Path.absname("test/fixtures/image_with_imagedescription_and_caption-abstract.jpg"), + tempfile: + Path.absname("test/fixtures/image_with_imagedescription_and_caption-abstract.jpg"), + description: "Some description" + } + + assert Filter.Exiftool.ReadDescription.filter(uploads) == + {:ok, :noop} + end + + test "otherwise returns ImageDescription when present" do + uploads_after = %Pleroma.Upload{ + name: "image_with_imagedescription_and_caption-abstract.jpg", + content_type: "image/jpeg", + path: Path.absname("test/fixtures/image_with_imagedescription_and_caption-abstract.jpg"), + tempfile: + Path.absname("test/fixtures/image_with_imagedescription_and_caption-abstract.jpg"), + description: "a descriptive white pixel" + } + + assert Filter.Exiftool.ReadDescription.filter(@uploads) == + {:ok, :filtered, uploads_after} + end + + test "otherwise returns iptc:Caption-Abstract when present" do + upload = %Pleroma.Upload{ + name: "image_with_caption-abstract.jpg", + content_type: "image/jpeg", + path: Path.absname("test/fixtures/image_with_caption-abstract.jpg"), + tempfile: Path.absname("test/fixtures/image_with_caption-abstract.jpg"), + description: nil + } + + upload_after = %Pleroma.Upload{ + name: "image_with_caption-abstract.jpg", + content_type: "image/jpeg", + path: Path.absname("test/fixtures/image_with_caption-abstract.jpg"), + tempfile: Path.absname("test/fixtures/image_with_caption-abstract.jpg"), + description: "an abstract white pixel" + } + + assert Filter.Exiftool.ReadDescription.filter(upload) == + {:ok, :filtered, upload_after} + end + + test "otherwise returns nil" do + uploads = %Pleroma.Upload{ + name: "image_with_no_description.jpg", + content_type: "image/jpeg", + path: Path.absname("test/fixtures/image_with_no_description.jpg"), + tempfile: Path.absname("test/fixtures/image_with_no_description.jpg"), + description: nil + } + + assert Filter.Exiftool.ReadDescription.filter(uploads) == + {:ok, :filtered, uploads} + end + + test "Return nil when image description from EXIF data exceeds the maximum length" do + clear_config([:instance, :description_limit], 5) + + assert Filter.Exiftool.ReadDescription.filter(@uploads) == + {:ok, :filtered, @uploads} + end + + test "Ignores content with only whitespace" do + uploads = %Pleroma.Upload{ + name: "non-existant.jpg", + content_type: "image/jpeg", + path: + Path.absname( + "test/fixtures/image_with_imagedescription_and_caption-abstract_whitespaces.jpg" + ), + tempfile: + Path.absname( + "test/fixtures/image_with_imagedescription_and_caption-abstract_whitespaces.jpg" + ), + description: nil + } + + assert Filter.Exiftool.ReadDescription.filter(uploads) == + {:ok, :filtered, uploads} + end + + test "Return nil when image description from EXIF data can't be read" do + uploads = %Pleroma.Upload{ + name: "non-existant.jpg", + content_type: "image/jpeg", + path: Path.absname("test/fixtures/non-existant.jpg"), + tempfile: Path.absname("test/fixtures/non-existant_tmp.jpg"), + description: nil + } + + assert Filter.Exiftool.ReadDescription.filter(uploads) == + {:ok, :filtered, uploads} + end +end diff --git a/test/pleroma/upload/filter/exiftool_test.exs b/test/pleroma/upload/filter/exiftool/strip_location_test.exs index 0a0ef2bdf..7e1541f60 100644 --- a/test/pleroma/upload/filter/exiftool_test.exs +++ b/test/pleroma/upload/filter/exiftool/strip_location_test.exs @@ -2,7 +2,7 @@ # Copyright © 2017-2022 Pleroma Authors <https://pleroma.social/> # SPDX-License-Identifier: AGPL-3.0-only -defmodule Pleroma.Upload.Filter.ExiftoolTest do +defmodule Pleroma.Upload.Filter.Exiftool.StripLocationTest do use Pleroma.DataCase, async: true alias Pleroma.Upload.Filter @@ -21,7 +21,7 @@ defmodule Pleroma.Upload.Filter.ExiftoolTest do tempfile: Path.absname("test/fixtures/DSCN0010_tmp.jpg") } - assert Filter.Exiftool.filter(upload) == {:ok, :filtered} + assert Filter.Exiftool.StripLocation.filter(upload) == {:ok, :filtered} {exif_original, 0} = System.cmd("exiftool", ["test/fixtures/DSCN0010.jpg"]) {exif_filtered, 0} = System.cmd("exiftool", ["test/fixtures/DSCN0010_tmp.jpg"]) @@ -37,6 +37,6 @@ defmodule Pleroma.Upload.Filter.ExiftoolTest do content_type: "image/webp" } - assert Filter.Exiftool.filter(upload) == {:ok, :noop} + assert Filter.Exiftool.StripLocation.filter(upload) == {:ok, :noop} end end diff --git a/test/pleroma/user_test.exs b/test/pleroma/user_test.exs index 884b846ae..408389c3a 100644 --- a/test/pleroma/user_test.exs +++ b/test/pleroma/user_test.exs @@ -618,9 +618,10 @@ defmodule Pleroma.UserTest do end test "it restricts certain nicknames" do + clear_config([User, :restricted_nicknames], ["about"]) [restricted_name | _] = Pleroma.Config.get([User, :restricted_nicknames]) - assert is_bitstring(restricted_name) + assert is_binary(restricted_name) params = @full_user_data @@ -631,6 +632,23 @@ defmodule Pleroma.UserTest do refute changeset.valid? end + test "it is case-insensitive when restricting nicknames" do + clear_config([User, :restricted_nicknames], ["about"]) + [restricted_name | _] = Pleroma.Config.get([User, :restricted_nicknames]) + + assert is_binary(restricted_name) + + restricted_upcase_name = String.upcase(restricted_name) + + params = + @full_user_data + |> Map.put(:nickname, restricted_upcase_name) + + changeset = User.register_changeset(%User{}, params) + + refute changeset.valid? + end + test "it blocks blacklisted email domains" do clear_config([User, :email_blacklist], ["trolling.world"]) @@ -639,6 +657,11 @@ defmodule Pleroma.UserTest do changeset = User.register_changeset(%User{}, params) refute changeset.valid? + # Block with case-insensitive match + params = Map.put(@full_user_data, :email, "troll@TrOlLing.wOrld") + changeset = User.register_changeset(%User{}, params) + refute changeset.valid? + # Block with subdomain match params = Map.put(@full_user_data, :email, "troll@gnomes.trolling.world") changeset = User.register_changeset(%User{}, params) diff --git a/test/pleroma/web/activity_pub/activity_pub_controller_test.exs b/test/pleroma/web/activity_pub/activity_pub_controller_test.exs index 1923ec479..1c5c40e84 100644 --- a/test/pleroma/web/activity_pub/activity_pub_controller_test.exs +++ b/test/pleroma/web/activity_pub/activity_pub_controller_test.exs @@ -291,6 +291,30 @@ defmodule Pleroma.Web.ActivityPub.ActivityPubControllerTest do assert json_response(conn, 200) == ObjectView.render("object.json", %{object: note}) end + test "does not cache authenticated response", %{conn: conn} do + user = insert(:user) + reader = insert(:user) + + {:ok, post} = + CommonAPI.post(user, %{status: "test @#{reader.nickname}", visibility: "local"}) + + object = Object.normalize(post, fetch: false) + uuid = String.split(object.data["id"], "/") |> List.last() + + assert response = + conn + |> assign(:user, reader) + |> put_req_header("accept", "application/activity+json") + |> get("/objects/#{uuid}") + + json_response(response, 200) + + conn + |> put_req_header("accept", "application/activity+json") + |> get("/objects/#{uuid}") + |> json_response(404) + end + test "it returns 404 for non-public messages", %{conn: conn} do note = insert(:direct_note) uuid = String.split(note.data["id"], "/") |> List.last() diff --git a/test/pleroma/web/activity_pub/mrf/anti_followbot_policy_test.exs b/test/pleroma/web/activity_pub/mrf/anti_followbot_policy_test.exs index 8192629c6..732a5a7f3 100644 --- a/test/pleroma/web/activity_pub/mrf/anti_followbot_policy_test.exs +++ b/test/pleroma/web/activity_pub/mrf/anti_followbot_policy_test.exs @@ -6,6 +6,7 @@ defmodule Pleroma.Web.ActivityPub.MRF.AntiFollowbotPolicyTest do use Pleroma.DataCase, async: true import Pleroma.Factory + alias Pleroma.User alias Pleroma.Web.ActivityPub.MRF.AntiFollowbotPolicy describe "blocking based on attributes" do @@ -38,21 +39,55 @@ defmodule Pleroma.Web.ActivityPub.MRF.AntiFollowbotPolicyTest do assert {:reject, "[AntiFollowbotPolicy]" <> _} = AntiFollowbotPolicy.filter(message) end + + test "matches followbots by actor_type" do + actor = insert(:user, %{actor_type: "Service"}) + target = insert(:user) + + message = %{ + "@context" => "https://www.w3.org/ns/activitystreams", + "type" => "Follow", + "actor" => actor.ap_id, + "object" => target.ap_id, + "id" => "https://example.com/activities/1234" + } + + assert {:reject, "[AntiFollowbotPolicy]" <> _} = AntiFollowbotPolicy.filter(message) + end end - test "it allows non-followbots" do - actor = insert(:user) - target = insert(:user) + describe "it allows" do + test "non-followbots" do + actor = insert(:user) + target = insert(:user) - message = %{ - "@context" => "https://www.w3.org/ns/activitystreams", - "type" => "Follow", - "actor" => actor.ap_id, - "object" => target.ap_id, - "id" => "https://example.com/activities/1234" - } + message = %{ + "@context" => "https://www.w3.org/ns/activitystreams", + "type" => "Follow", + "actor" => actor.ap_id, + "object" => target.ap_id, + "id" => "https://example.com/activities/1234" + } - {:ok, _} = AntiFollowbotPolicy.filter(message) + {:ok, _} = AntiFollowbotPolicy.filter(message) + end + + test "bots if the target follows the bots" do + actor = insert(:user, %{actor_type: "Service"}) + target = insert(:user) + + User.follow(target, actor) + + message = %{ + "@context" => "https://www.w3.org/ns/activitystreams", + "type" => "Follow", + "actor" => actor.ap_id, + "object" => target.ap_id, + "id" => "https://example.com/activities/1234" + } + + {:ok, _} = AntiFollowbotPolicy.filter(message) + end end test "it gracefully handles nil display names" do diff --git a/test/pleroma/web/activity_pub/mrf/steal_emoji_policy_test.exs b/test/pleroma/web/activity_pub/mrf/steal_emoji_policy_test.exs index c537670f2..89d32352f 100644 --- a/test/pleroma/web/activity_pub/mrf/steal_emoji_policy_test.exs +++ b/test/pleroma/web/activity_pub/mrf/steal_emoji_policy_test.exs @@ -60,7 +60,7 @@ defmodule Pleroma.Web.ActivityPub.MRF.StealEmojiPolicyTest do |> File.exists?() end - test "reject shortcode", %{message: message} do + test "reject regex shortcode", %{message: message} do refute "firedfox" in installed() clear_config(:mrf_steal_emoji, @@ -74,6 +74,20 @@ defmodule Pleroma.Web.ActivityPub.MRF.StealEmojiPolicyTest do refute "firedfox" in installed() end + test "reject string shortcode", %{message: message} do + refute "firedfox" in installed() + + clear_config(:mrf_steal_emoji, + hosts: ["example.org"], + size_limit: 284_468, + rejected_shortcodes: ["firedfox"] + ) + + assert {:ok, _message} = StealEmojiPolicy.filter(message) + + refute "firedfox" in installed() + end + test "reject if size is above the limit", %{message: message} do refute "firedfox" in installed() diff --git a/test/pleroma/web/activity_pub/object_validators/attachment_validator_test.exs b/test/pleroma/web/activity_pub/object_validators/attachment_validator_test.exs index b07011b76..9a17e277e 100644 --- a/test/pleroma/web/activity_pub/object_validators/attachment_validator_test.exs +++ b/test/pleroma/web/activity_pub/object_validators/attachment_validator_test.exs @@ -27,6 +27,46 @@ defmodule Pleroma.Web.ActivityPub.ObjectValidators.AttachmentValidatorTest do assert attachment.mediaType == "application/octet-stream" end + test "works with an unknown but valid mime type" do + attachment = %{ + "mediaType" => "x-custom/x-type", + "type" => "Document", + "url" => "https://example.org" + } + + assert {:ok, attachment} = + AttachmentValidator.cast_and_validate(attachment) + |> Ecto.Changeset.apply_action(:insert) + + assert attachment.mediaType == "x-custom/x-type" + end + + test "works with invalid mime types" do + attachment = %{ + "mediaType" => "x-customx-type", + "type" => "Document", + "url" => "https://example.org" + } + + assert {:ok, attachment} = + AttachmentValidator.cast_and_validate(attachment) + |> Ecto.Changeset.apply_action(:insert) + + assert attachment.mediaType == "application/octet-stream" + + attachment = %{ + "mediaType" => "https://example.org", + "type" => "Document", + "url" => "https://example.org" + } + + assert {:ok, attachment} = + AttachmentValidator.cast_and_validate(attachment) + |> Ecto.Changeset.apply_action(:insert) + + assert attachment.mediaType == "application/octet-stream" + end + test "it turns mastodon attachments into our attachments" do attachment = %{ "url" => diff --git a/test/pleroma/web/admin_api/controllers/announcement_controller_test.exs b/test/pleroma/web/admin_api/controllers/announcement_controller_test.exs new file mode 100644 index 000000000..5b8148c05 --- /dev/null +++ b/test/pleroma/web/admin_api/controllers/announcement_controller_test.exs @@ -0,0 +1,281 @@ +# Pleroma: A lightweight social networking server +# Copyright © 2017-2022 Pleroma Authors <https://pleroma.social/> +# SPDX-License-Identifier: AGPL-3.0-only + +defmodule Pleroma.Web.AdminAPI.AnnouncementControllerTest do + use Pleroma.Web.ConnCase + + import Pleroma.Factory + + setup do + admin = insert(:user, is_admin: true) + token = insert(:oauth_admin_token, user: admin) + + conn = + build_conn() + |> assign(:user, admin) + |> assign(:token, token) + + {:ok, %{admin: admin, token: token, conn: conn}} + end + + describe "GET /api/v1/pleroma/admin/announcements" do + test "it lists all announcements", %{conn: conn} do + %{id: id} = insert(:announcement) + + response = + conn + |> get("/api/v1/pleroma/admin/announcements") + |> json_response_and_validate_schema(:ok) + + assert [%{"id" => ^id}] = response + end + + test "it paginates announcements", %{conn: conn} do + _announcements = Enum.map(0..20, fn _ -> insert(:announcement) end) + + response = + conn + |> get("/api/v1/pleroma/admin/announcements") + |> json_response_and_validate_schema(:ok) + + assert length(response) == 20 + end + + test "it paginates announcements with custom params", %{conn: conn} do + announcements = Enum.map(0..20, fn _ -> insert(:announcement) end) + + response = + conn + |> get("/api/v1/pleroma/admin/announcements", limit: 5, offset: 7) + |> json_response_and_validate_schema(:ok) + + assert length(response) == 5 + assert Enum.at(response, 0)["id"] == Enum.at(announcements, 7).id + end + + test "it returns empty list with out-of-bounds offset", %{conn: conn} do + _announcements = Enum.map(0..20, fn _ -> insert(:announcement) end) + + response = + conn + |> get("/api/v1/pleroma/admin/announcements", offset: 21) + |> json_response_and_validate_schema(:ok) + + assert [] = response + end + + test "it rejects invalid pagination params", %{conn: conn} do + conn + |> get("/api/v1/pleroma/admin/announcements", limit: 0) + |> json_response_and_validate_schema(400) + + conn + |> get("/api/v1/pleroma/admin/announcements", limit: -1) + |> json_response_and_validate_schema(400) + + conn + |> get("/api/v1/pleroma/admin/announcements", offset: -1) + |> json_response_and_validate_schema(400) + end + end + + describe "GET /api/v1/pleroma/admin/announcements/:id" do + test "it displays one announcement", %{conn: conn} do + %{id: id} = insert(:announcement) + + response = + conn + |> get("/api/v1/pleroma/admin/announcements/#{id}") + |> json_response_and_validate_schema(:ok) + + assert %{"id" => ^id} = response + end + + test "it returns not found for non-existent id", %{conn: conn} do + %{id: id} = insert(:announcement) + + _response = + conn + |> get("/api/v1/pleroma/admin/announcements/#{id}xxx") + |> json_response_and_validate_schema(:not_found) + end + end + + describe "DELETE /api/v1/pleroma/admin/announcements/:id" do + test "it deletes specified announcement", %{conn: conn} do + %{id: id} = insert(:announcement) + + _response = + conn + |> delete("/api/v1/pleroma/admin/announcements/#{id}") + |> json_response_and_validate_schema(:ok) + end + + test "it returns not found for non-existent id", %{conn: conn} do + %{id: id} = insert(:announcement) + + _response = + conn + |> delete("/api/v1/pleroma/admin/announcements/#{id}xxx") + |> json_response_and_validate_schema(:not_found) + + assert %{id: ^id} = Pleroma.Announcement.get_by_id(id) + end + end + + describe "PATCH /api/v1/pleroma/admin/announcements/:id" do + test "it returns not found for non-existent id", %{conn: conn} do + %{id: id} = insert(:announcement) + + _response = + conn + |> put_req_header("content-type", "application/json") + |> patch("/api/v1/pleroma/admin/announcements/#{id}xxx", %{}) + |> json_response_and_validate_schema(:not_found) + + assert %{id: ^id} = Pleroma.Announcement.get_by_id(id) + end + + test "it updates a field", %{conn: conn} do + %{id: id} = insert(:announcement) + + now = NaiveDateTime.utc_now() |> NaiveDateTime.truncate(:second) + starts_at = NaiveDateTime.add(now, -10, :second) + + _response = + conn + |> put_req_header("content-type", "application/json") + |> patch("/api/v1/pleroma/admin/announcements/#{id}", %{ + starts_at: NaiveDateTime.to_iso8601(starts_at) + }) + |> json_response_and_validate_schema(:ok) + + new = Pleroma.Announcement.get_by_id(id) + + assert NaiveDateTime.compare(new.starts_at, starts_at) == :eq + end + + test "it updates with time with utc timezone", %{conn: conn} do + %{id: id} = insert(:announcement) + + now = DateTime.now("Etc/UTC") |> elem(1) |> DateTime.truncate(:second) + starts_at = DateTime.add(now, -10, :second) + + _response = + conn + |> put_req_header("content-type", "application/json") + |> patch("/api/v1/pleroma/admin/announcements/#{id}", %{ + starts_at: DateTime.to_iso8601(starts_at) + }) + |> json_response_and_validate_schema(:ok) + + new = Pleroma.Announcement.get_by_id(id) + + assert DateTime.compare(new.starts_at, starts_at) == :eq + end + + test "it updates a data field", %{conn: conn} do + %{id: id} = announcement = insert(:announcement, data: %{"all_day" => true}) + + assert announcement.data["all_day"] == true + + new_content = "new content" + + response = + conn + |> put_req_header("content-type", "application/json") + |> patch("/api/v1/pleroma/admin/announcements/#{id}", %{ + content: new_content + }) + |> json_response_and_validate_schema(:ok) + + assert response["content"] == new_content + assert response["all_day"] == true + + new = Pleroma.Announcement.get_by_id(id) + + assert new.data["content"] == new_content + assert new.data["all_day"] == true + end + + test "it nullifies a nullable field", %{conn: conn} do + now = NaiveDateTime.utc_now() |> NaiveDateTime.truncate(:second) + starts_at = NaiveDateTime.add(now, -10, :second) + + %{id: id} = insert(:announcement, starts_at: starts_at) + + response = + conn + |> put_req_header("content-type", "application/json") + |> patch("/api/v1/pleroma/admin/announcements/#{id}", %{ + starts_at: nil + }) + |> json_response_and_validate_schema(:ok) + + assert response["starts_at"] == nil + + new = Pleroma.Announcement.get_by_id(id) + + assert new.starts_at == nil + end + end + + describe "POST /api/v1/pleroma/admin/announcements" do + test "it creates an announcement", %{conn: conn} do + content = "test post announcement api" + + now = NaiveDateTime.utc_now() |> NaiveDateTime.truncate(:second) + starts_at = NaiveDateTime.add(now, -10, :second) + ends_at = NaiveDateTime.add(now, 10, :second) + + response = + conn + |> put_req_header("content-type", "application/json") + |> post("/api/v1/pleroma/admin/announcements", %{ + "content" => content, + "starts_at" => NaiveDateTime.to_iso8601(starts_at), + "ends_at" => NaiveDateTime.to_iso8601(ends_at), + "all_day" => true + }) + |> json_response_and_validate_schema(:ok) + + assert %{"content" => ^content, "all_day" => true} = response + + announcement = Pleroma.Announcement.get_by_id(response["id"]) + + assert not is_nil(announcement) + + assert NaiveDateTime.compare(announcement.starts_at, starts_at) == :eq + assert NaiveDateTime.compare(announcement.ends_at, ends_at) == :eq + end + + test "creating with time with utc timezones", %{conn: conn} do + content = "test post announcement api" + + now = DateTime.now("Etc/UTC") |> elem(1) |> DateTime.truncate(:second) + starts_at = DateTime.add(now, -10, :second) + ends_at = DateTime.add(now, 10, :second) + + response = + conn + |> put_req_header("content-type", "application/json") + |> post("/api/v1/pleroma/admin/announcements", %{ + "content" => content, + "starts_at" => DateTime.to_iso8601(starts_at), + "ends_at" => DateTime.to_iso8601(ends_at), + "all_day" => true + }) + |> json_response_and_validate_schema(:ok) + + assert %{"content" => ^content, "all_day" => true} = response + + announcement = Pleroma.Announcement.get_by_id(response["id"]) + + assert not is_nil(announcement) + + assert DateTime.compare(announcement.starts_at, starts_at) == :eq + assert DateTime.compare(announcement.ends_at, ends_at) == :eq + end + end +end diff --git a/test/pleroma/web/admin_api/controllers/report_controller_test.exs b/test/pleroma/web/admin_api/controllers/report_controller_test.exs index 6211956c6..30dcb87e2 100644 --- a/test/pleroma/web/admin_api/controllers/report_controller_test.exs +++ b/test/pleroma/web/admin_api/controllers/report_controller_test.exs @@ -355,7 +355,6 @@ defmodule Pleroma.Web.AdminAPI.ReportControllerTest do } = note end - @tag :erratic test "it returns reports with notes", %{conn: conn, admin: admin} do conn = get(conn, "/api/pleroma/admin/reports") @@ -364,7 +363,8 @@ defmodule Pleroma.Web.AdminAPI.ReportControllerTest do [note, _] = notes assert note["user"]["nickname"] == admin.nickname - assert note["content"] == "this is disgusting!" + # We use '=~' because the order of the notes isn't guaranteed + assert note["content"] =~ "this is disgusting" assert note["created_at"] assert response["total"] == 1 end diff --git a/test/pleroma/web/gettext_test.exs b/test/pleroma/web/gettext_test.exs index 9ede4827e..e186f1ab3 100644 --- a/test/pleroma/web/gettext_test.exs +++ b/test/pleroma/web/gettext_test.exs @@ -38,6 +38,17 @@ defmodule Pleroma.Web.GettextTest do end end + test "putting en locale at the front should not make gettext fallback unexpectedly" do + Pleroma.Web.Gettext.with_locales ["en", "en_test"] do + assert "Your account is awaiting approval" == + Pleroma.Web.Gettext.dpgettext( + "static_pages", + "approval pending email subject", + "Your account is awaiting approval" + ) + end + end + test "duplicated locale in list should not result in infinite loops" do Pleroma.Web.Gettext.with_locales ["x_unsupported", "x_unsupported", "en_test"] do assert "xxYour account is awaiting approvalxx" == diff --git a/test/pleroma/web/mastodon_api/controllers/announcement_controller_test.exs b/test/pleroma/web/mastodon_api/controllers/announcement_controller_test.exs new file mode 100644 index 000000000..60c9978c2 --- /dev/null +++ b/test/pleroma/web/mastodon_api/controllers/announcement_controller_test.exs @@ -0,0 +1,169 @@ +# Pleroma: A lightweight social networking server +# Copyright © 2017-2022 Pleroma Authors <https://pleroma.social/> +# SPDX-License-Identifier: AGPL-3.0-only + +defmodule Pleroma.Web.MastodonAPI.AnnouncementControllerTest do + use Pleroma.Web.ConnCase + + import Pleroma.Factory + + alias Pleroma.Announcement + alias Pleroma.AnnouncementReadRelationship + + describe "GET /api/v1/announcements" do + setup do + %{conn: conn} = oauth_access([]) + {:ok, conn: conn} + end + + test "it does not allow guests", %{conn: conn} do + _response = + conn + |> assign(:token, nil) + |> get("/api/v1/announcements") + |> json_response_and_validate_schema(:forbidden) + end + + test "it allows users with scopes" do + %{conn: conn} = oauth_access(["read:accounts"]) + + _response = + conn + |> get("/api/v1/announcements") + |> json_response_and_validate_schema(:ok) + end + + test "it lists all announcements", %{conn: conn} do + %{id: id} = insert(:announcement) + + response = + conn + |> get("/api/v1/announcements") + |> json_response_and_validate_schema(:ok) + + assert [%{"id" => ^id}] = response + end + + test "it returns time with utc timezone", %{conn: conn} do + start_time = + NaiveDateTime.utc_now() + |> NaiveDateTime.add(-999_999, :second) + |> NaiveDateTime.truncate(:second) + + end_time = + NaiveDateTime.utc_now() + |> NaiveDateTime.add(999_999, :second) + |> NaiveDateTime.truncate(:second) + + %{id: id} = insert(:announcement, %{starts_at: start_time, ends_at: end_time}) + + response = + conn + |> get("/api/v1/announcements") + |> json_response_and_validate_schema(:ok) + + assert [%{"id" => ^id}] = [announcement] = response + + assert String.ends_with?(announcement["starts_at"], "Z") + assert String.ends_with?(announcement["ends_at"], "Z") + end + + test "it does not list announcements starting after current time", %{conn: conn} do + time = NaiveDateTime.utc_now() |> NaiveDateTime.add(999_999, :second) + insert(:announcement, starts_at: time) + + response = + conn + |> get("/api/v1/announcements") + |> json_response_and_validate_schema(:ok) + + assert [] = response + end + + test "it does not list announcements ending before current time", %{conn: conn} do + time = NaiveDateTime.utc_now() |> NaiveDateTime.add(-999_999, :second) + insert(:announcement, ends_at: time) + + response = + conn + |> get("/api/v1/announcements") + |> json_response_and_validate_schema(:ok) + + assert [] = response + end + + test "when authenticated, also expose read property", %{conn: conn} do + %{id: id} = insert(:announcement) + + response = + conn + |> get("/api/v1/announcements") + |> json_response_and_validate_schema(:ok) + + assert [%{"id" => ^id, "read" => false}] = response + end + + test "when authenticated and announcement is read by user" do + %{id: id} = announcement = insert(:announcement) + user = insert(:user) + + AnnouncementReadRelationship.mark_read(user, announcement) + + %{conn: conn} = oauth_access(["read:accounts"], user: user) + + response = + conn + |> get("/api/v1/announcements") + |> json_response_and_validate_schema(:ok) + + assert [%{"id" => ^id, "read" => true}] = response + end + end + + describe "POST /api/v1/announcements/:id/dismiss" do + setup do: oauth_access(["write:accounts"]) + + test "it requires auth", %{conn: conn} do + %{id: id} = insert(:announcement) + + _response = + conn + |> assign(:token, nil) + |> post("/api/v1/announcements/#{id}/dismiss") + |> json_response_and_validate_schema(:forbidden) + end + + test "it requires write:accounts oauth scope" do + %{id: id} = insert(:announcement) + + %{conn: conn} = oauth_access(["read:accounts"]) + + _response = + conn + |> post("/api/v1/announcements/#{id}/dismiss") + |> json_response_and_validate_schema(:forbidden) + end + + test "it gives 404 for non-existent announcements", %{conn: conn} do + %{id: id} = insert(:announcement) + + _response = + conn + |> post("/api/v1/announcements/#{id}xxx/dismiss") + |> json_response_and_validate_schema(:not_found) + end + + test "it marks announcement as read", %{user: user, conn: conn} do + %{id: id} = announcement = insert(:announcement) + + refute Announcement.read_by?(announcement, user) + + _response = + conn + |> post("/api/v1/announcements/#{id}/dismiss") + |> json_response_and_validate_schema(:ok) + + assert Announcement.read_by?(announcement, user) + end + end +end diff --git a/test/pleroma/web/plugs/cache_test.exs b/test/pleroma/web/plugs/cache_test.exs index d84e2d268..0c119528d 100644 --- a/test/pleroma/web/plugs/cache_test.exs +++ b/test/pleroma/web/plugs/cache_test.exs @@ -179,4 +179,22 @@ defmodule Pleroma.Web.Plugs.CacheTest do |> send_resp(:im_a_teapot, "🥤") |> sent_resp() end + + test "ignores if skip_cache is assigned" do + assert @miss_resp == + conn(:get, "/") + |> assign(:skip_cache, true) + |> Cache.call(%{query_params: false, ttl: nil}) + |> put_resp_content_type("cofe/hot") + |> send_resp(:ok, "cofe") + |> sent_resp() + + assert @miss_resp == + conn(:get, "/") + |> assign(:skip_cache, true) + |> Cache.call(%{query_params: false, ttl: nil}) + |> put_resp_content_type("cofe/hot") + |> send_resp(:ok, "cofe") + |> sent_resp() + end end diff --git a/test/pleroma/web/twitter_api/remote_follow_controller_test.exs b/test/pleroma/web/twitter_api/remote_follow_controller_test.exs index a493dc9f6..2b57a42a4 100644 --- a/test/pleroma/web/twitter_api/remote_follow_controller_test.exs +++ b/test/pleroma/web/twitter_api/remote_follow_controller_test.exs @@ -410,4 +410,49 @@ defmodule Pleroma.Web.TwitterAPI.RemoteFollowControllerTest do assert response =~ "Error following account" end end + + describe "avatar url" do + test "without media proxy" do + clear_config([:media_proxy, :enabled], false) + + user = + insert(:user, %{ + local: false, + avatar: %{"url" => [%{"href" => "https://remote.org/avatar.png"}]} + }) + + avatar_url = Pleroma.Web.TwitterAPI.RemoteFollowView.avatar_url(user) + + assert avatar_url == "https://remote.org/avatar.png" + end + + test "with media proxy" do + clear_config([:media_proxy, :enabled], true) + + user = + insert(:user, %{ + local: false, + avatar: %{"url" => [%{"href" => "https://remote.org/avatar.png"}]} + }) + + avatar_url = Pleroma.Web.TwitterAPI.RemoteFollowView.avatar_url(user) + url = Pleroma.Web.Endpoint.url() + + assert String.starts_with?(avatar_url, url) + end + + test "local avatar is not proxied" do + clear_config([:media_proxy, :enabled], true) + + user = + insert(:user, %{ + local: true, + avatar: %{"url" => [%{"href" => "#{Pleroma.Web.Endpoint.url()}/localuser/avatar.png"}]} + }) + + avatar_url = Pleroma.Web.TwitterAPI.RemoteFollowView.avatar_url(user) + + assert avatar_url == "#{Pleroma.Web.Endpoint.url()}/localuser/avatar.png" + end + end end diff --git a/test/support/factory.ex b/test/support/factory.ex index 09456debf..efbf3df2e 100644 --- a/test/support/factory.ex +++ b/test/support/factory.ex @@ -627,4 +627,16 @@ defmodule Pleroma.Factory do context: ["home"] } end + + def announcement_factory(params \\ %{}) do + data = Map.get(params, :data, %{}) + + {_, params} = Map.pop(params, :data) + + %Pleroma.Announcement{ + data: Map.merge(%{"content" => "test announcement", "all_day" => false}, data) + } + |> Map.merge(params) + |> Pleroma.Announcement.add_rendered_properties() + end end |
