Bump version

updated-dependencies:
- dependency-name: requests
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

.dockerignore

@@ -5,7 +5,6 @@
 !/bookmarks
 !/siteroot
 
-!/background-tasks-wrapper.sh
 !/bootstrap.sh
 !/LICENSE.txt
 !/manage.py

.github/workflows/main.yaml

@@ -24,7 +24,9 @@ jobs:
       - name: Install Node dependencies
         run: npm ci
       - name: Setup Python environment
-        run: pip install -r requirements.txt -r requirements.dev.txt
+        run: |
+          pip install -r requirements.txt -r requirements.dev.txt
+          mkdir data
       - name: Run tests
         run: python manage.py test bookmarks.tests
   e2e_tests:
@@ -47,6 +49,7 @@ jobs:
         run: |
           pip install -r requirements.txt -r requirements.dev.txt
           playwright install chromium
+          mkdir data
       - name: Run build
         run: |
           npm run build

.gitignore

@@ -191,3 +191,8 @@ typings/
 /tmp
 # Database file
 /data
+# ublock + chromium
+/uBlock0.chromium
+/chromium-profile
+# direnv
+/.direnv

CHANGELOG.md

@@ -1,5 +1,80 @@
 # Changelog
 
+## v1.30.0 (20/04/2024)
+
+### What's Changed
+* Add reader mode by @sissbruecker in https://github.com/sissbruecker/linkding/pull/703
+* Allow uploading custom files for bookmarks by @sissbruecker in https://github.com/sissbruecker/linkding/pull/713
+* Add option for marking bookmarks as unread by default by @ab623 in https://github.com/sissbruecker/linkding/pull/706
+* Make blocking cookie banners more reliable by @sissbruecker in https://github.com/sissbruecker/linkding/pull/699
+* Close bookmark details with escape by @sissbruecker in https://github.com/sissbruecker/linkding/pull/702
+* Show proper name for bookmark assets in admin by @ab623 in https://github.com/sissbruecker/linkding/pull/708
+* Bump sqlparse from 0.4.4 to 0.5.0 by @dependabot in https://github.com/sissbruecker/linkding/pull/704
+
+### New Contributors
+* @ab623 made their first contribution in https://github.com/sissbruecker/linkding/pull/706
+
+**Full Changelog**: https://github.com/sissbruecker/linkding/compare/v1.29.0...v1.30.0
+
+---
+
+## v1.29.0 (14/04/2024)
+
+### What's Changed
+* Remove ads and cookie banners from HTML snapshots by @sissbruecker in https://github.com/sissbruecker/linkding/pull/695
+* Add button for creating missing HTML snapshots by @sissbruecker in https://github.com/sissbruecker/linkding/pull/696
+* Refresh file list when there are queued snapshots by @sissbruecker in https://github.com/sissbruecker/linkding/pull/697
+* Bump idna from 3.6 to 3.7 by @dependabot in https://github.com/sissbruecker/linkding/pull/694
+
+
+**Full Changelog**: https://github.com/sissbruecker/linkding/compare/v1.28.0...v1.29.0
+
+---
+
+## v1.28.0 (09/04/2024)
+
+### What's Changed
+* Add option to disable SSL verification for OIDC by @akaSyntaax in https://github.com/sissbruecker/linkding/pull/684
+* Add full backup method by @sissbruecker in https://github.com/sissbruecker/linkding/pull/686
+* Truncate snapshot filename for long URLs by @sissbruecker in https://github.com/sissbruecker/linkding/pull/687
+* Add option for customizing single-file timeout by @pettijohn in https://github.com/sissbruecker/linkding/pull/688
+* Add option for passing arguments to single-file command by @pettijohn in https://github.com/sissbruecker/linkding/pull/691
+* Fix typo by @tianheg in https://github.com/sissbruecker/linkding/pull/689
+
+### New Contributors
+* @akaSyntaax made their first contribution in https://github.com/sissbruecker/linkding/pull/684
+* @pettijohn made their first contribution in https://github.com/sissbruecker/linkding/pull/688
+
+**Full Changelog**: https://github.com/sissbruecker/linkding/compare/v1.27.1...v1.28.0
+
+---
+
+## v1.27.1 (07/04/2024)
+
+### What's Changed
+* Fix HTML snapshot errors related to single-file-cli by @sissbruecker in https://github.com/sissbruecker/linkding/pull/683
+* Replace django-background-tasks with huey by @sissbruecker in https://github.com/sissbruecker/linkding/pull/657
+* Add Authelia OIDC example to docs by @hugo-vrijswijk in https://github.com/sissbruecker/linkding/pull/675
+
+
+**Full Changelog**: https://github.com/sissbruecker/linkding/compare/v1.27.0...v1.27.1
+
+---
+
+## v1.27.0 (01/04/2024)
+
+### What's Changed
+* Archive snapshots of websites locally by @sissbruecker in https://github.com/sissbruecker/linkding/pull/672
+* Add Railway hosting option by @tianheg in https://github.com/sissbruecker/linkding/pull/661
+* Add how to for increasing the font size by @sissbruecker in https://github.com/sissbruecker/linkding/pull/667
+
+### New Contributors
+* @tianheg made their first contribution in https://github.com/sissbruecker/linkding/pull/661
+
+**Full Changelog**: https://github.com/sissbruecker/linkding/compare/v1.26.0...v1.27.0
+
+---
+
 ## v1.26.0 (30/03/2024)
 
 ### What's Changed

README.md

@@ -45,7 +45,7 @@ The name comes from:
 - Admin panel for user self-service and raw data access
 
 
-**Demo:** https://demo.linkding.link/ ([see here](https://github.com/sissbruecker/linkding/issues/408) if you have trouble accessing it)
+**Demo:** https://demo.linkding.link/
 
 **Screenshot:**
 

background-tasks-wrapper.sh

@@ -1,5 +0,0 @@
-#!/usr/bin/env bash
-# Wrapper script used by supervisord to first clear task locks before starting the background task processor
-
-python manage.py clean_tasks
-exec python manage.py process_tasks

bookmarks/admin.py

@@ -1,22 +1,96 @@
-from background_task.admin import TaskAdmin, CompletedTaskAdmin
-from background_task.models import Task, CompletedTask
 from django.contrib import admin, messages
 from django.contrib.admin import AdminSite
 from django.contrib.auth.admin import UserAdmin
 from django.contrib.auth.models import User
+from django.core.paginator import Paginator
 from django.db.models import Count, QuerySet
+from django.shortcuts import render
+from django.urls import path
 from django.utils.translation import ngettext, gettext
+from huey.contrib.djhuey import HUEY as huey
 from rest_framework.authtoken.admin import TokenAdmin
 from rest_framework.authtoken.models import TokenProxy
 
-from bookmarks.models import Bookmark, Tag, UserProfile, Toast, FeedToken
+from bookmarks.models import Bookmark, BookmarkAsset, Tag, UserProfile, Toast, FeedToken
 from bookmarks.services.bookmarks import archive_bookmark, unarchive_bookmark
 
 
+# Custom paginator to paginate through Huey tasks
+class TaskPaginator(Paginator):
+    def __init__(self):
+        super().__init__(self, 100)
+        self.task_count = huey.storage.queue_size()
+
+    @property
+    def count(self):
+        return self.task_count
+
+    def page(self, number):
+        limit = self.per_page
+        offset = (number - 1) * self.per_page
+        return self._get_page(
+            self.enqueued_items(limit, offset),
+            number,
+            self,
+        )
+
+    # Copied from Huey's SqliteStorage with some modifications to allow pagination
+    def enqueued_items(self, limit, offset):
+        to_bytes = lambda b: bytes(b) if not isinstance(b, bytes) else b
+        sql = "select data from task where queue=? order by priority desc, id limit ? offset ?"
+        params = (huey.storage.name, limit, offset)
+
+        serialized_tasks = [
+            to_bytes(i) for i, in huey.storage.sql(sql, params, results=True)
+        ]
+        return [huey.deserialize_task(task) for task in serialized_tasks]
+
+
+# Custom view to display Huey tasks in the admin
+def background_task_view(request):
+    page_number = int(request.GET.get("p", 1))
+    paginator = TaskPaginator()
+    page = paginator.get_page(page_number)
+    page_range = paginator.get_elided_page_range(page_number, on_each_side=2, on_ends=2)
+    context = {
+        **linkding_admin_site.each_context(request),
+        "title": "Background tasks",
+        "page": page,
+        "page_range": page_range,
+        "tasks": page.object_list,
+    }
+    return render(request, "admin/background_tasks.html", context)
+
+
 class LinkdingAdminSite(AdminSite):
     site_header = "linkding administration"
     site_title = "linkding Admin"
 
+    def get_urls(self):
+        urls = super().get_urls()
+        custom_urls = [
+            path("tasks/", background_task_view, name="background_tasks"),
+        ]
+        return custom_urls + urls
+
+    def get_app_list(self, request, app_label=None):
+        app_list = super().get_app_list(request, app_label)
+        app_list += [
+            {
+                "name": "Huey",
+                "app_label": "huey_app",
+                "models": [
+                    {
+                        "name": "Queued tasks",
+                        "object_name": "background_tasks",
+                        "admin_url": "/admin/tasks/",
+                        "view_only": True,
+                    }
+                ],
+            }
+        ]
+        return app_list
+
 
 class AdminBookmark(admin.ModelAdmin):
     list_display = ("resolved_title", "url", "is_archived", "owner", "date_added")
@@ -125,6 +199,19 @@ class AdminBookmark(admin.ModelAdmin):
         )
 
 
+class AdminBookmarkAsset(admin.ModelAdmin):
+    @admin.display(description="Display Name")
+    def custom_display_name(self, obj):
+        return str(obj)
+
+    list_display = ("custom_display_name", "date_created", "status")
+    search_fields = (
+        "custom_display_name",
+        "file",
+    )
+    list_filter = ("status",)
+
+
 class AdminTag(admin.ModelAdmin):
     list_display = ("name", "bookmarks_count", "owner", "date_added")
     search_fields = ("name", "owner__username")
@@ -200,10 +287,9 @@ class AdminFeedToken(admin.ModelAdmin):
 
 linkding_admin_site = LinkdingAdminSite()
 linkding_admin_site.register(Bookmark, AdminBookmark)
+linkding_admin_site.register(BookmarkAsset, AdminBookmarkAsset)
 linkding_admin_site.register(Tag, AdminTag)
 linkding_admin_site.register(User, AdminCustomUser)
 linkding_admin_site.register(TokenProxy, TokenAdmin)
 linkding_admin_site.register(Toast, AdminToast)
 linkding_admin_site.register(FeedToken, AdminFeedToken)
-linkding_admin_site.register(Task, TaskAdmin)
-linkding_admin_site.register(CompletedTask, CompletedTaskAdmin)

bookmarks/api/routes.py

@@ -11,6 +11,7 @@ from bookmarks.api.serializers import (
     UserProfileSerializer,
 )
 from bookmarks.models import Bookmark, BookmarkSearch, Tag, User
+from bookmarks.services import auto_tagging
 from bookmarks.services.bookmarks import (
     archive_bookmark,
     unarchive_bookmark,
@@ -99,13 +100,26 @@ class BookmarkViewSet(
         # Either return metadata from existing bookmark, or scrape from URL
         if bookmark:
             metadata = WebsiteMetadata(
-                url, bookmark.website_title, bookmark.website_description
+                url,
+                bookmark.website_title,
+                bookmark.website_description,
+                None,
             )
         else:
             metadata = website_loader.load_website_metadata(url)
 
+        # Return tags that would be automatically applied to the bookmark
+        profile = request.user.profile
+        auto_tags = []
+        if profile.auto_tagging_rules:
+            auto_tags = auto_tagging.get_tags(profile.auto_tagging_rules, url)
+
         return Response(
-            {"bookmark": existing_bookmark_data, "metadata": metadata.to_dict()},
+            {
+                "bookmark": existing_bookmark_data,
+                "metadata": metadata.to_dict(),
+                "auto_tags": auto_tags,
+            },
             status=status.HTTP_200_OK,
         )
 

bookmarks/e2e/e2e_test_bookmark_details_modal.py

@@ -1,3 +1,4 @@
+from django.test import override_settings
 from django.urls import reverse
 from playwright.sync_api import sync_playwright, expect
 
@@ -33,6 +34,11 @@ class BookmarkDetailsModalE2ETestCase(LinkdingE2ETestCase):
             overlay.click(position={"x": 0, "y": 0})
             expect(details_modal).to_be_hidden()
 
+            # close with escape
+            details_modal = self.open_details_modal(bookmark)
+            self.page.keyboard.press("Escape")
+            expect(details_modal).to_be_hidden()
+
     def test_toggle_archived(self):
         bookmark = self.setup_bookmark()
 
@@ -44,14 +50,17 @@ class BookmarkDetailsModalE2ETestCase(LinkdingE2ETestCase):
             details_modal = self.open_details_modal(bookmark)
             details_modal.get_by_text("Archived", exact=False).click()
             expect(self.locate_bookmark(bookmark.title)).not_to_be_visible()
+            self.assertReloads(0)
 
             # unarchive
             url = reverse("bookmarks:archived")
             self.page.goto(self.live_server_url + url)
+            self.resetReloads()
 
             details_modal = self.open_details_modal(bookmark)
             details_modal.get_by_text("Archived", exact=False).click()
             expect(self.locate_bookmark(bookmark.title)).not_to_be_visible()
+            self.assertReloads(0)
 
     def test_toggle_unread(self):
         bookmark = self.setup_bookmark()
@@ -66,11 +75,13 @@ class BookmarkDetailsModalE2ETestCase(LinkdingE2ETestCase):
             details_modal.get_by_text("Unread").click()
             bookmark_item = self.locate_bookmark(bookmark.title)
             expect(bookmark_item.get_by_text("Unread")).to_be_visible()
+            self.assertReloads(0)
 
             # mark as read
             details_modal.get_by_text("Unread").click()
             bookmark_item = self.locate_bookmark(bookmark.title)
             expect(bookmark_item.get_by_text("Unread")).not_to_be_visible()
+            self.assertReloads(0)
 
     def test_toggle_shared(self):
         profile = self.get_or_create_test_user().profile
@@ -89,11 +100,13 @@ class BookmarkDetailsModalE2ETestCase(LinkdingE2ETestCase):
             details_modal.get_by_text("Shared").click()
             bookmark_item = self.locate_bookmark(bookmark.title)
             expect(bookmark_item.get_by_text("Shared")).to_be_visible()
+            self.assertReloads(0)
 
             # unshare bookmark
             details_modal.get_by_text("Shared").click()
             bookmark_item = self.locate_bookmark(bookmark.title)
             expect(bookmark_item.get_by_text("Shared")).not_to_be_visible()
+            self.assertReloads(0)
 
     def test_edit_return_url(self):
         bookmark = self.setup_bookmark()
@@ -131,3 +144,33 @@ class BookmarkDetailsModalE2ETestCase(LinkdingE2ETestCase):
             expect(self.locate_bookmark(bookmark.title)).not_to_be_visible()
 
         self.assertEqual(Bookmark.objects.count(), 0)
+
+    @override_settings(LD_ENABLE_SNAPSHOTS=True)
+    def test_create_snapshot_remove_snapshot(self):
+        bookmark = self.setup_bookmark()
+
+        with sync_playwright() as p:
+            url = reverse("bookmarks:index") + f"?q={bookmark.title}"
+            self.open(url, p)
+
+            details_modal = self.open_details_modal(bookmark)
+            asset_list = details_modal.locator(".assets")
+
+            # No snapshots initially
+            snapshot = asset_list.get_by_text("HTML snapshot from", exact=False)
+            expect(snapshot).not_to_be_visible()
+
+            # Create snapshot
+            details_modal.get_by_text("Create HTML snapshot", exact=False).click()
+            self.assertReloads(0)
+
+            # Has new snapshots
+            expect(snapshot).to_be_visible()
+
+            # Create snapshot
+            asset_list.get_by_text("Remove", exact=False).click()
+            asset_list.get_by_text("Confirm", exact=False).click()
+
+            # Snapshot is removed
+            expect(snapshot).not_to_be_visible()
+            self.assertReloads(0)

bookmarks/e2e/e2e_test_bookmark_form.py

@@ -85,3 +85,25 @@ class BookmarkFormE2ETestCase(LinkdingE2ETestCase):
 
             page.get_by_label("URL").fill(bookmark.url)
             expect(details).to_have_attribute("open", value="")
+
+    def test_create_should_preview_auto_tags(self):
+        profile = self.get_or_create_test_user().profile
+        profile.auto_tagging_rules = "github.com dev github"
+        profile.save()
+
+        with sync_playwright() as p:
+            # Open page with URL that should have auto tags
+            browser = self.setup_browser(p)
+            page = browser.new_page()
+            url = self.live_server_url + reverse("bookmarks:new")
+            url += f"?url=https%3A%2F%2Fgithub.com%2Fsissbruecker%2Flinkding"
+            page.goto(url)
+
+            auto_tags_hint = page.locator(".form-input-hint.auto-tags")
+            expect(auto_tags_hint).to_be_visible()
+            expect(auto_tags_hint).to_have_text("Auto tags: dev github")
+
+            # Change to URL without auto tags
+            page.get_by_label("URL").fill("https://example.com")
+
+            expect(auto_tags_hint).to_be_hidden()

bookmarks/e2e/e2e_test_bookmark_page_bulk_edit.py

@@ -194,7 +194,7 @@ class BookmarkPagePartialUpdatesE2ETestCase(LinkdingE2ETestCase):
 
             self.locate_bulk_edit_toggle().click()
 
-            checkboxes = page.locator("label[ld-bulk-edit-checkbox] input")
+            checkboxes = page.locator("label.bulk-edit-checkbox input")
             self.assertEqual(6, checkboxes.count())
             for i in range(checkboxes.count()):
                 expect(checkboxes.nth(i)).not_to_be_checked()
@@ -264,13 +264,13 @@ class BookmarkPagePartialUpdatesE2ETestCase(LinkdingE2ETestCase):
 
             # Hide select across by toggling a single bookmark
             self.locate_bookmark("Bookmark 1").locator(
-                "label[ld-bulk-edit-checkbox]"
+                "label.bulk-edit-checkbox"
             ).click()
             expect(self.locate_bulk_edit_select_across()).not_to_be_visible()
 
             # Show select across again, verify it is unchecked
             self.locate_bookmark("Bookmark 1").locator(
-                "label[ld-bulk-edit-checkbox]"
+                "label.bulk-edit-checkbox"
             ).click()
             expect(self.locate_bulk_edit_select_across()).not_to_be_checked()
 
@@ -297,7 +297,7 @@ class BookmarkPagePartialUpdatesE2ETestCase(LinkdingE2ETestCase):
             expect(bookmark_list).not_to_be_visible()
 
             # Verify bulk edit checkboxes are reset
-            checkboxes = page.locator("label[ld-bulk-edit-checkbox] input")
+            checkboxes = page.locator("label.bulk-edit-checkbox input")
             self.assertEqual(31, checkboxes.count())
             for i in range(checkboxes.count()):
                 expect(checkboxes.nth(i)).not_to_be_checked()

bookmarks/e2e/e2e_test_bookmark_page_partial_updates.py

@@ -169,7 +169,7 @@ class BookmarkPagePartialUpdatesE2ETestCase(LinkdingE2ETestCase):
 
             self.locate_bulk_edit_toggle().click()
             self.locate_bookmark("Bookmark 2").locator(
-                "label[ld-bulk-edit-checkbox]"
+                "label.bulk-edit-checkbox"
             ).click()
             self.select_bulk_action("Archive")
             self.locate_bulk_edit_bar().get_by_text("Execute").click()
@@ -187,7 +187,7 @@ class BookmarkPagePartialUpdatesE2ETestCase(LinkdingE2ETestCase):
 
             self.locate_bulk_edit_toggle().click()
             self.locate_bookmark("Bookmark 2").locator(
-                "label[ld-bulk-edit-checkbox]"
+                "label.bulk-edit-checkbox"
             ).click()
             self.select_bulk_action("Delete")
             self.locate_bulk_edit_bar().get_by_text("Execute").click()
@@ -230,7 +230,7 @@ class BookmarkPagePartialUpdatesE2ETestCase(LinkdingE2ETestCase):
 
             self.locate_bulk_edit_toggle().click()
             self.locate_bookmark("Archived Bookmark 2").locator(
-                "label[ld-bulk-edit-checkbox]"
+                "label.bulk-edit-checkbox"
             ).click()
             self.select_bulk_action("Unarchive")
             self.locate_bulk_edit_bar().get_by_text("Execute").click()
@@ -248,7 +248,7 @@ class BookmarkPagePartialUpdatesE2ETestCase(LinkdingE2ETestCase):
 
             self.locate_bulk_edit_toggle().click()
             self.locate_bookmark("Archived Bookmark 2").locator(
-                "label[ld-bulk-edit-checkbox]"
+                "label.bulk-edit-checkbox"
             ).click()
             self.select_bulk_action("Delete")
             self.locate_bulk_edit_bar().get_by_text("Execute").click()

bookmarks/e2e/e2e_test_tag_cloud_modal.py

@@ -0,0 +1,76 @@
+from django.test import override_settings
+from django.urls import reverse
+from playwright.sync_api import sync_playwright, expect, Locator
+
+from bookmarks.e2e.helpers import LinkdingE2ETestCase
+from bookmarks.models import Bookmark
+
+
+class TagCloudModalE2ETestCase(LinkdingE2ETestCase):
+    def test_show_modal_close_modal(self):
+        self.setup_bookmark(tags=[self.setup_tag(name="cooking")])
+        self.setup_bookmark(tags=[self.setup_tag(name="hiking")])
+
+        with sync_playwright() as p:
+            page = self.open(reverse("bookmarks:index"), p)
+
+            # use smaller viewport to make tags button visible
+            page.set_viewport_size({"width": 375, "height": 812})
+
+            # open tag cloud modal
+            modal_trigger = page.locator(".content-area-header").get_by_role(
+                "button", name="Tags"
+            )
+            modal_trigger.click()
+
+            # verify modal is visible
+            modal = page.locator(".modal")
+            expect(modal).to_be_visible()
+            expect(modal.locator(".modal-title")).to_have_text("Tags")
+
+            # close with close button
+            modal.locator("button.close").click()
+            expect(modal).to_be_hidden()
+
+            # open modal again
+            modal_trigger.click()
+
+            # close with backdrop
+            backdrop = modal.locator(".modal-overlay")
+            backdrop.click(position={"x": 0, "y": 0})
+            expect(modal).to_be_hidden()
+
+    def test_select_tag(self):
+        self.setup_bookmark(tags=[self.setup_tag(name="cooking")])
+        self.setup_bookmark(tags=[self.setup_tag(name="hiking")])
+
+        with sync_playwright() as p:
+            page = self.open(reverse("bookmarks:index"), p)
+
+            # use smaller viewport to make tags button visible
+            page.set_viewport_size({"width": 375, "height": 812})
+
+            # open tag cloud modal
+            modal_trigger = page.locator(".content-area-header").get_by_role(
+                "button", name="Tags"
+            )
+            modal_trigger.click()
+
+            # verify tags are displayed
+            modal = page.locator(".modal")
+            unselected_tags = modal.locator(".unselected-tags")
+            expect(unselected_tags.get_by_text("cooking")).to_be_visible()
+            expect(unselected_tags.get_by_text("hiking")).to_be_visible()
+
+            # select tag
+            unselected_tags.get_by_text("cooking").click()
+
+            # open modal again
+            modal_trigger.click()
+
+            # verify tag is selected, other tag is not visible anymore
+            selected_tags = modal.locator(".selected-tags")
+            expect(selected_tags.get_by_text("cooking")).to_be_visible()
+
+            expect(unselected_tags.get_by_text("cooking")).not_to_be_visible()
+            expect(unselected_tags.get_by_text("hiking")).not_to_be_visible()

bookmarks/e2e/helpers.py

@@ -39,6 +39,9 @@ class LinkdingE2ETestCase(LiveServerTestCase, BookmarkFactoryMixin):
     def assertReloads(self, count: int):
         self.assertEqual(self.num_loads, count)
 
+    def resetReloads(self):
+        self.num_loads = 0
+
     def locate_bookmark_list(self):
         return self.page.locator("ul[ld-bookmark-list]")
 
@@ -62,7 +65,7 @@ class LinkdingE2ETestCase(LiveServerTestCase, BookmarkFactoryMixin):
         return self.page.locator(".bulk-edit-bar")
 
     def locate_bulk_edit_select_all(self):
-        return self.locate_bulk_edit_bar().locator("label[ld-bulk-edit-checkbox][all]")
+        return self.locate_bulk_edit_bar().locator("label.bulk-edit-checkbox.all")
 
     def locate_bulk_edit_select_across(self):
         return self.locate_bulk_edit_bar().locator("label.select-across")

bookmarks/frontend/behaviors/bookmark-page.js

@@ -1,67 +1,8 @@
-import { registerBehavior, swapContent } from "./index";
+import { Behavior, registerBehavior } from "./index";
 
-class BookmarkPage {
+class BookmarkItem extends Behavior {
   constructor(element) {
-    this.element = element;
-    this.form = element.querySelector("form.bookmark-actions");
-    this.form.addEventListener("submit", this.onFormSubmit.bind(this));
-
-    this.bookmarkList = element.querySelector(".bookmark-list-container");
-    this.tagCloud = element.querySelector(".tag-cloud-container");
-
-    document.addEventListener("bookmark-page-refresh", () => {
-      this.refresh();
-    });
-  }
-
-  async onFormSubmit(event) {
-    event.preventDefault();
-
-    const url = this.form.action;
-    const formData = new FormData(this.form);
-    formData.append(event.submitter.name, event.submitter.value);
-
-    await fetch(url, {
-      method: "POST",
-      body: formData,
-      redirect: "manual", // ignore redirect
-    });
-    await this.refresh();
-  }
-
-  async refresh() {
-    const query = window.location.search;
-    const bookmarksUrl = this.element.getAttribute("bookmarks-url");
-    const tagsUrl = this.element.getAttribute("tags-url");
-    Promise.all([
-      fetch(`${bookmarksUrl}${query}`).then((response) => response.text()),
-      fetch(`${tagsUrl}${query}`).then((response) => response.text()),
-    ]).then(([bookmarkListHtml, tagCloudHtml]) => {
-      swapContent(this.bookmarkList, bookmarkListHtml);
-      swapContent(this.tagCloud, tagCloudHtml);
-
-      // Dispatch list updated event
-      const listElement = this.bookmarkList.querySelector(
-        "ul[data-bookmarks-total]",
-      );
-      const bookmarksTotal =
-        (listElement && listElement.dataset.bookmarksTotal) || 0;
-
-      this.bookmarkList.dispatchEvent(
-        new CustomEvent("bookmark-list-updated", {
-          bubbles: true,
-          detail: { bookmarksTotal },
-        }),
-      );
-    });
-  }
-}
-
-registerBehavior("ld-bookmark-page", BookmarkPage);
-
-class BookmarkItem {
-  constructor(element) {
-    this.element = element;
+    super(element);
 
     // Toggle notes
     const notesToggle = element.querySelector(".toggle-notes");
@@ -72,9 +13,11 @@ class BookmarkItem {
     // Add tooltip to title if it is truncated
     const titleAnchor = element.querySelector(".title > a");
     const titleSpan = titleAnchor.querySelector("span");
-    if (titleSpan.offsetWidth > titleAnchor.offsetWidth) {
-      titleAnchor.dataset.tooltip = titleSpan.textContent;
-    }
+    requestAnimationFrame(() => {
+      if (titleSpan.offsetWidth > titleAnchor.offsetWidth) {
+        titleAnchor.dataset.tooltip = titleSpan.textContent;
+      }
+    });
   }
 
   onToggleNotes(event) {

bookmarks/frontend/behaviors/bulk-edit.js

@@ -1,46 +1,60 @@
-import { registerBehavior } from "./index";
+import { Behavior, registerBehavior } from "./index";
 
-class BulkEdit {
+class BulkEdit extends Behavior {
   constructor(element) {
-    this.element = element;
+    super(element);
+
     this.active = false;
-    this.actionSelect = element.querySelector("select[name='bulk_action']");
-    this.tagAutoComplete = element.querySelector(".tag-autocomplete");
-    this.selectAcross = element.querySelector("label.select-across");
 
-    element.addEventListener(
-      "bulk-edit-toggle-active",
-      this.onToggleActive.bind(this),
-    );
-    element.addEventListener(
-      "bulk-edit-toggle-all",
-      this.onToggleAll.bind(this),
-    );
-    element.addEventListener(
-      "bulk-edit-toggle-bookmark",
-      this.onToggleBookmark.bind(this),
-    );
-    element.addEventListener(
-      "bookmark-list-updated",
-      this.onListUpdated.bind(this),
-    );
+    this.onToggleActive = this.onToggleActive.bind(this);
+    this.onToggleAll = this.onToggleAll.bind(this);
+    this.onToggleBookmark = this.onToggleBookmark.bind(this);
+    this.onActionSelected = this.onActionSelected.bind(this);
 
-    this.actionSelect.addEventListener(
-      "change",
-      this.onActionSelected.bind(this),
-    );
+    this.init();
+    // Reset when bookmarks are refreshed
+    document.addEventListener("refresh-bookmark-list-done", () => this.init());
   }
 
-  get allCheckbox() {
-    return this.element.querySelector("[ld-bulk-edit-checkbox][all] input");
-  }
+  init() {
+    // Update elements
+    this.activeToggle = this.element.querySelector(".bulk-edit-active-toggle");
+    this.actionSelect = this.element.querySelector(
+      "select[name='bulk_action']",
+    );
+    this.tagAutoComplete = this.element.querySelector(".tag-autocomplete");
+    this.selectAcross = this.element.querySelector("label.select-across");
+    this.allCheckbox = this.element.querySelector(
+      ".bulk-edit-checkbox.all input",
+    );
+    this.bookmarkCheckboxes = Array.from(
+      this.element.querySelectorAll(".bulk-edit-checkbox:not(.all) input"),
+    );
 
-  get bookmarkCheckboxes() {
-    return [
-      ...this.element.querySelectorAll(
-        "[ld-bulk-edit-checkbox]:not([all]) input",
-      ),
-    ];
+    // Remove previous listeners if elements are the same
+    this.activeToggle.removeEventListener("click", this.onToggleActive);
+    this.actionSelect.removeEventListener("change", this.onActionSelected);
+    this.allCheckbox.removeEventListener("change", this.onToggleAll);
+    this.bookmarkCheckboxes.forEach((checkbox) => {
+      checkbox.removeEventListener("change", this.onToggleBookmark);
+    });
+
+    // Reset checkbox states
+    this.reset();
+
+    // Update total number of bookmarks
+    const totalHolder = this.element.querySelector("[data-bookmarks-total]");
+    const total = totalHolder?.dataset.bookmarksTotal || 0;
+    const totalSpan = this.selectAcross.querySelector("span.total");
+    totalSpan.textContent = total;
+
+    // Add new listeners
+    this.activeToggle.addEventListener("click", this.onToggleActive);
+    this.actionSelect.addEventListener("change", this.onActionSelected);
+    this.allCheckbox.addEventListener("change", this.onToggleAll);
+    this.bookmarkCheckboxes.forEach((checkbox) => {
+      checkbox.addEventListener("change", this.onToggleBookmark);
+    });
   }
 
   onToggleActive() {
@@ -81,16 +95,6 @@ class BulkEdit {
     }
   }
 
-  onListUpdated(event) {
-    // Reset checkbox states
-    this.reset();
-
-    // Update total number of bookmarks
-    const total = event.detail.bookmarksTotal;
-    const totalSpan = this.selectAcross.querySelector("span.total");
-    totalSpan.textContent = total;
-  }
-
   updateSelectAcross(allChecked) {
     if (allChecked) {
       this.selectAcross.classList.remove("d-none");
@@ -109,33 +113,4 @@ class BulkEdit {
   }
 }
 
-class BulkEditActiveToggle {
-  constructor(element) {
-    this.element = element;
-    element.addEventListener("click", this.onClick.bind(this));
-  }
-
-  onClick() {
-    this.element.dispatchEvent(
-      new CustomEvent("bulk-edit-toggle-active", { bubbles: true }),
-    );
-  }
-}
-
-class BulkEditCheckbox {
-  constructor(element) {
-    this.element = element;
-    element.addEventListener("change", this.onChange.bind(this));
-  }
-
-  onChange() {
-    const type = this.element.hasAttribute("all") ? "all" : "bookmark";
-    this.element.dispatchEvent(
-      new CustomEvent(`bulk-edit-toggle-${type}`, { bubbles: true }),
-    );
-  }
-}
-
 registerBehavior("ld-bulk-edit", BulkEdit);
-registerBehavior("ld-bulk-edit-active-toggle", BulkEditActiveToggle);
-registerBehavior("ld-bulk-edit-checkbox", BulkEditCheckbox);

bookmarks/frontend/behaviors/confirm-button.js

@@ -1,25 +1,29 @@
-import { registerBehavior } from "./index";
+import { Behavior, registerBehavior } from "./index";
 
-class ConfirmButtonBehavior {
+class ConfirmButtonBehavior extends Behavior {
   constructor(element) {
-    const button = element;
-    button.dataset.type = button.type;
-    button.dataset.name = button.name;
-    button.dataset.value = button.value;
-    button.removeAttribute("type");
-    button.removeAttribute("name");
-    button.removeAttribute("value");
-    button.addEventListener("click", this.onClick.bind(this));
-    this.button = button;
+    super(element);
+    element.dataset.type = element.type;
+    element.dataset.name = element.name;
+    element.dataset.value = element.value;
+    element.removeAttribute("type");
+    element.removeAttribute("name");
+    element.removeAttribute("value");
+    element.addEventListener("click", this.onClick.bind(this));
+  }
+
+  destroy() {
+    Behavior.interacting = false;
   }
 
   onClick(event) {
     event.preventDefault();
+    Behavior.interacting = true;
 
     const container = document.createElement("span");
     container.className = "confirmation";
 
-    const icon = this.button.getAttribute("confirm-icon");
+    const icon = this.element.getAttribute("ld-confirm-icon");
     if (icon) {
       const iconElement = document.createElementNS(
         "http://www.w3.org/2000/svg",
@@ -31,27 +35,27 @@ class ConfirmButtonBehavior {
       container.append(iconElement);
     }
 
-    const question = this.button.getAttribute("confirm-question");
+    const question = this.element.getAttribute("ld-confirm-question");
     if (question) {
       const questionElement = document.createElement("span");
       questionElement.innerText = question;
       container.append(question);
     }
 
-    const buttonClasses = Array.from(this.button.classList.values())
+    const buttonClasses = Array.from(this.element.classList.values())
       .filter((cls) => cls.startsWith("btn"))
       .join(" ");
 
-    const cancelButton = document.createElement(this.button.nodeName);
+    const cancelButton = document.createElement(this.element.nodeName);
     cancelButton.type = "button";
     cancelButton.innerText = question ? "No" : "Cancel";
     cancelButton.className = `${buttonClasses} mr-1`;
     cancelButton.addEventListener("click", this.reset.bind(this));
 
-    const confirmButton = document.createElement(this.button.nodeName);
-    confirmButton.type = this.button.dataset.type;
-    confirmButton.name = this.button.dataset.name;
-    confirmButton.value = this.button.dataset.value;
+    const confirmButton = document.createElement(this.element.nodeName);
+    confirmButton.type = this.element.dataset.type;
+    confirmButton.name = this.element.dataset.name;
+    confirmButton.value = this.element.dataset.value;
     confirmButton.innerText = question ? "Yes" : "Confirm";
     confirmButton.className = buttonClasses;
     confirmButton.addEventListener("click", this.reset.bind(this));
@@ -59,14 +63,15 @@ class ConfirmButtonBehavior {
     container.append(cancelButton, confirmButton);
     this.container = container;
 
-    this.button.before(container);
-    this.button.classList.add("d-none");
+    this.element.before(container);
+    this.element.classList.add("d-none");
   }
 
   reset() {
     setTimeout(() => {
+      Behavior.interacting = false;
       this.container.remove();
-      this.button.classList.remove("d-none");
+      this.element.classList.remove("d-none");
     });
   }
 }

bookmarks/frontend/behaviors/dropdown.js

@@ -1,8 +1,8 @@
-import { registerBehavior } from "./index";
+import { Behavior, registerBehavior } from "./index";
 
-class DropdownBehavior {
+class DropdownBehavior extends Behavior {
   constructor(element) {
-    this.element = element;
+    super(element);
     this.opened = false;
     this.onOutsideClick = this.onOutsideClick.bind(this);
 

bookmarks/frontend/behaviors/fetch.js

@@ -0,0 +1,48 @@
+import { Behavior, fireEvents, registerBehavior, swap } from "./index";
+
+class FetchBehavior extends Behavior {
+  constructor(element) {
+    super(element);
+
+    const eventName = element.getAttribute("ld-on");
+    const interval = parseInt(element.getAttribute("ld-interval")) * 1000;
+
+    this.onFetch = this.onFetch.bind(this);
+    this.onInterval = this.onInterval.bind(this);
+
+    element.addEventListener(eventName, this.onFetch);
+    if (interval) {
+      this.intervalId = setInterval(this.onInterval, interval);
+    }
+  }
+
+  destroy() {
+    if (this.intervalId) {
+      clearInterval(this.intervalId);
+    }
+  }
+
+  async onFetch(maybeEvent) {
+    if (maybeEvent) {
+      maybeEvent.preventDefault();
+    }
+    const url = this.element.getAttribute("ld-fetch");
+    const html = await fetch(url).then((response) => response.text());
+
+    const target = this.element.getAttribute("ld-target");
+    const select = this.element.getAttribute("ld-select");
+    swap(this.element, html, { target, select });
+
+    const events = this.element.getAttribute("ld-fire");
+    fireEvents(events);
+  }
+
+  onInterval() {
+    if (Behavior.interacting) {
+      return;
+    }
+    this.onFetch();
+  }
+}
+
+registerBehavior("ld-fetch", FetchBehavior);

bookmarks/frontend/behaviors/form.js

@@ -1,12 +1,13 @@
-import { registerBehavior, swap } from "./index";
+import { Behavior, fireEvents, registerBehavior } from "./index";
 
-class FormBehavior {
+class FormBehavior extends Behavior {
   constructor(element) {
-    this.element = element;
-    element.addEventListener("submit", this.onFormSubmit.bind(this));
+    super(element);
+
+    element.addEventListener("submit", this.onSubmit.bind(this));
   }
 
-  async onFormSubmit(event) {
+  async onSubmit(event) {
     event.preventDefault();
 
     const url = this.element.action;
@@ -21,34 +22,43 @@ class FormBehavior {
       redirect: "manual", // ignore redirect
     });
 
-    // Dispatch refresh events
-    const refreshEvents = this.element.getAttribute("refresh-events");
-    if (refreshEvents) {
-      refreshEvents.split(",").forEach((eventName) => {
-        document.dispatchEvent(new CustomEvent(eventName));
-      });
+    const events = this.element.getAttribute("ld-fire");
+    if (fireEvents) {
+      fireEvents(events);
     }
-
-    // Refresh form
-    await this.refresh();
-  }
-
-  async refresh() {
-    const refreshUrl = this.element.getAttribute("refresh-url");
-    const html = await fetch(refreshUrl).then((response) => response.text());
-    swap(this.element, html);
   }
 }
 
-class FormAutoSubmitBehavior {
+class AutoSubmitBehavior extends Behavior {
   constructor(element) {
-    this.element = element;
-    this.element.addEventListener("change", () => {
-      const form = this.element.closest("form");
+    super(element);
+
+    element.addEventListener("change", () => {
+      const form = element.closest("form");
       form.dispatchEvent(new Event("submit", { cancelable: true }));
     });
   }
 }
 
+class UploadButton extends Behavior {
+  constructor(element) {
+    super(element);
+
+    const fileInput = element.nextElementSibling;
+
+    element.addEventListener("click", () => {
+      fileInput.click();
+    });
+
+    fileInput.addEventListener("change", () => {
+      const form = fileInput.closest("form");
+      const event = new Event("submit", { cancelable: true });
+      event.submitter = element;
+      form.dispatchEvent(event);
+    });
+  }
+}
+
 registerBehavior("ld-form", FormBehavior);
-registerBehavior("ld-form-auto-submit", FormAutoSubmitBehavior);
+registerBehavior("ld-auto-submit", AutoSubmitBehavior);
+registerBehavior("ld-upload-button", UploadButton);

bookmarks/frontend/behaviors/global-shortcuts.js

@@ -1,7 +1,9 @@
-import { registerBehavior } from "./index";
+import { Behavior, registerBehavior } from "./index";
+
+class GlobalShortcuts extends Behavior {
+  constructor(element) {
+    super(element);
 
-class GlobalShortcuts {
-  constructor() {
     document.addEventListener("keydown", this.onKeyDown.bind(this));
   }
 

bookmarks/frontend/behaviors/index.js

@@ -1,4 +1,35 @@
 const behaviorRegistry = {};
+const debug = false;
+
+const mutationObserver = new MutationObserver((mutations) => {
+  mutations.forEach((mutation) => {
+    mutation.removedNodes.forEach((node) => {
+      if (node instanceof HTMLElement && !node.isConnected) {
+        destroyBehaviors(node);
+      }
+    });
+    mutation.addedNodes.forEach((node) => {
+      if (node instanceof HTMLElement && node.isConnected) {
+        applyBehaviors(node);
+      }
+    });
+  });
+});
+
+mutationObserver.observe(document.body, {
+  childList: true,
+  subtree: true,
+});
+
+export class Behavior {
+  constructor(element) {
+    this.element = element;
+  }
+
+  destroy() {}
+}
+
+Behavior.interacting = false;
 
 export function registerBehavior(name, behavior) {
   behaviorRegistry[name] = behavior;
@@ -33,18 +64,82 @@ export function applyBehaviors(container, behaviorNames = null) {
 
       const behaviorInstance = new behavior(element);
       element.__behaviors.push(behaviorInstance);
+      if (debug) {
+        console.log(
+          `[Behavior] ${behaviorInstance.constructor.name} initialized`,
+        );
+      }
     });
   });
 }
 
-export function swap(element, html) {
-  const dom = new DOMParser().parseFromString(html, "text/html");
-  const newElement = dom.body.firstChild;
-  element.replaceWith(newElement);
-  applyBehaviors(newElement);
+export function destroyBehaviors(element) {
+  const behaviorNames = Object.keys(behaviorRegistry);
+
+  behaviorNames.forEach((behaviorName) => {
+    const elements = Array.from(element.querySelectorAll(`[${behaviorName}]`));
+    elements.push(element);
+
+    elements.forEach((element) => {
+      if (!element.__behaviors) {
+        return;
+      }
+
+      element.__behaviors.forEach((behavior) => {
+        behavior.destroy();
+        if (debug) {
+          console.log(`[Behavior] ${behavior.constructor.name} destroyed`);
+        }
+      });
+      delete element.__behaviors;
+    });
+  });
 }
 
-export function swapContent(element, html) {
-  element.innerHTML = html;
-  applyBehaviors(element);
+export function swap(element, html, options) {
+  const dom = new DOMParser().parseFromString(html, "text/html");
+
+  let targetElement = element;
+  let strategy = "innerHTML";
+  if (options.target) {
+    const parts = options.target.split("|");
+    targetElement =
+      parts[0] === "self" ? element : document.querySelector(parts[0]);
+    strategy = parts[1] || "innerHTML";
+  }
+
+  let contents = Array.from(dom.body.children);
+  if (options.select) {
+    contents = Array.from(dom.querySelectorAll(options.select));
+  }
+
+  switch (strategy) {
+    case "append":
+      targetElement.append(...contents);
+      break;
+    case "outerHTML":
+      targetElement.parentElement.replaceChild(contents[0], targetElement);
+      break;
+    case "innerHTML":
+    default:
+      Array.from(targetElement.children).forEach((child) => {
+        child.remove();
+      });
+      targetElement.append(...contents);
+  }
+}
+
+export function fireEvents(events) {
+  if (!events) {
+    return;
+  }
+  events.split(",").forEach((eventName) => {
+    const targets = Array.from(
+      document.querySelectorAll(`[ld-on='${eventName}']`),
+    );
+    targets.push(document);
+    targets.forEach((target) => {
+      target.dispatchEvent(new CustomEvent(eventName));
+    });
+  });
 }

bookmarks/frontend/behaviors/modal.js

@@ -1,97 +1,48 @@
-import { applyBehaviors, registerBehavior } from "./index";
+import { Behavior, registerBehavior } from "./index";
 
-class ModalBehavior {
+class ModalBehavior extends Behavior {
   constructor(element) {
-    const toggle = element;
-    toggle.addEventListener("click", this.onToggleClick.bind(this));
-    this.toggle = toggle;
+    super(element);
+
+    this.onClose = this.onClose.bind(this);
+    this.onKeyDown = this.onKeyDown.bind(this);
+
+    const modalOverlay = element.querySelector(".modal-overlay");
+    const closeButton = element.querySelector("button.close");
+    modalOverlay.addEventListener("click", this.onClose);
+    closeButton.addEventListener("click", this.onClose);
+
+    document.addEventListener("keydown", this.onKeyDown);
   }
 
-  async onToggleClick(event) {
-    // Ignore Ctrl + click
-    if (event.ctrlKey || event.metaKey) {
-      return;
-    }
-    event.preventDefault();
-    event.stopPropagation();
+  destroy() {
+    document.removeEventListener("keydown", this.onKeyDown);
+  }
 
-    // Create modal either by teleporting existing content or fetching from URL
-    const modal = this.toggle.hasAttribute("modal-content")
-      ? this.createFromContent()
-      : await this.createFromUrl();
+  onKeyDown(event) {
+    // Skip if event occurred within an input element
+    const targetNodeName = event.target.nodeName;
+    const isInputTarget =
+      targetNodeName === "INPUT" ||
+      targetNodeName === "SELECT" ||
+      targetNodeName === "TEXTAREA";
 
-    if (!modal) {
+    if (isInputTarget) {
       return;
     }
 
-    // Register close handlers
-    const modalOverlay = modal.querySelector(".modal-overlay");
-    const closeButton = modal.querySelector("button.close");
-    modalOverlay.addEventListener("click", this.onClose.bind(this));
-    closeButton.addEventListener("click", this.onClose.bind(this));
-
-    document.body.append(modal);
-    applyBehaviors(document.body);
-    this.modal = modal;
-  }
-
-  async createFromUrl() {
-    const url = this.toggle.getAttribute("modal-url");
-    const modalHtml = await fetch(url).then((response) => response.text());
-    const parser = new DOMParser();
-    const doc = parser.parseFromString(modalHtml, "text/html");
-    return doc.querySelector(".modal");
-  }
-
-  createFromContent() {
-    const contentSelector = this.toggle.getAttribute("modal-content");
-    const content = document.querySelector(contentSelector);
-    if (!content) {
-      return;
+    if (event.key === "Escape") {
+      event.preventDefault();
+      this.onClose();
     }
-
-    // Todo: make title configurable, only used for tag cloud for now
-    const modal = document.createElement("div");
-    modal.classList.add("modal", "active");
-    modal.innerHTML = `
-      <div class="modal-overlay" aria-label="Close"></div>
-      <div class="modal-container">
-        <div class="modal-header d-flex justify-between align-center">
-          <div class="modal-title h5">Tags</div>
-          <button class="close">
-          <svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" stroke-width="2" stroke="currentColor" fill="none" stroke-linecap="round" stroke-linejoin="round">
-            <path stroke="none" d="M0 0h24v24H0z" fill="none"></path>
-            <path d="M18 6l-12 12"></path>
-            <path d="M6 6l12 12"></path>
-          </svg>
-          </button>
-        </div>
-        <div class="modal-body">
-          <div class="content"></div>
-        </div>
-      </div>
-    `;
-
-    const contentOwner = content.parentElement;
-    const contentContainer = modal.querySelector(".content");
-    contentContainer.append(content);
-    this.content = content;
-    this.contentOwner = contentOwner;
-
-    return modal;
   }
 
   onClose() {
-    // Teleport content back
-    if (this.content && this.contentOwner) {
-      this.contentOwner.append(this.content);
-    }
-
-    // Remove modal
-    this.modal.classList.add("closing");
-    this.modal.addEventListener("animationend", (event) => {
+    document.removeEventListener("keydown", this.onKeyDown);
+    this.element.classList.add("closing");
+    this.element.addEventListener("animationend", (event) => {
       if (event.animationName === "fade-out") {
-        this.modal.remove();
+        this.element.remove();
       }
     });
   }

bookmarks/frontend/behaviors/tag-autocomplete.js

@@ -1,9 +1,10 @@
-import { registerBehavior } from "./index";
+import { Behavior, registerBehavior } from "./index";
 import TagAutoCompleteComponent from "../components/TagAutocomplete.svelte";
 import { ApiClient } from "../api";
 
-class TagAutocomplete {
+class TagAutocomplete extends Behavior {
   constructor(element) {
+    super(element);
     const wrapper = document.createElement("div");
     const apiBaseUrl = document.documentElement.dataset.apiBaseUrl || "";
     const apiClient = new ApiClient(apiBaseUrl);

bookmarks/frontend/components/TagAutocomplete.svelte

@@ -150,17 +150,27 @@
         display: block;
     }
 
+    .form-autocomplete-input {
+        box-sizing: border-box;
+        height: var(--control-size);
+        min-height: var(--control-size);
+        padding: 0;
+    }
+
+    .form-autocomplete-input input {
+        width: 100%;
+        height: 100%;
+        border: none;
+        margin: 0;
+    }
+
     .form-autocomplete.small .form-autocomplete-input {
         height: var(--control-size-sm);
         min-height: var(--control-size-sm);
-        padding: 0.05rem 0.3rem;
     }
 
     .form-autocomplete.small .form-autocomplete-input input {
-        width: 100%;
-        height: 100%;
-        margin: 0;
-        padding: 0;
+        padding: 0.05rem 0.3rem;
         font-size: var(--font-size-sm);
     }
 

bookmarks/frontend/index.js

@@ -2,6 +2,7 @@ import "./behaviors/bookmark-page";
 import "./behaviors/bulk-edit";
 import "./behaviors/confirm-button";
 import "./behaviors/dropdown";
+import "./behaviors/fetch";
 import "./behaviors/form";
 import "./behaviors/modal";
 import "./behaviors/global-shortcuts";

bookmarks/management/commands/backup.py

@@ -24,3 +24,8 @@ class Command(BaseCommand):
         source_db.close()
 
         self.stdout.write(self.style.SUCCESS(f"Backup created at {destination}"))
+        self.stdout.write(
+            self.style.WARNING(
+                "This backup method is deprecated and may be removed in the future. Please use the full_backup command instead, which creates backup zip file with all contents of the data folder."
+            )
+        )

bookmarks/management/commands/clean_tasks.py

@@ -1,15 +0,0 @@
-from background_task.models import Task, CompletedTask
-from django.core.management.base import BaseCommand
-
-
-class Command(BaseCommand):
-    help = "Remove task locks and clear completed task history"
-
-    def handle(self, *args, **options):
-        # Remove task locks
-        # If the background task processor exited while executing tasks, these tasks would still be marked as locked,
-        # even though no process is working on them, and would prevent the task processor from picking the next task in
-        # the queue
-        Task.objects.all().update(locked_by=None, locked_at=None)
-        # Clear task history to prevent them from bloating the DB
-        CompletedTask.objects.all().delete()

bookmarks/management/commands/full_backup.py

@@ -0,0 +1,75 @@
+import sqlite3
+import os
+import tempfile
+import zipfile
+
+from django.core.management.base import BaseCommand
+
+
+class Command(BaseCommand):
+    help = "Creates a backup of the linkding data folder"
+
+    def add_arguments(self, parser):
+        parser.add_argument("backup_file", type=str, help="Backup zip file destination")
+
+    def handle(self, *args, **options):
+        backup_file = options["backup_file"]
+        with zipfile.ZipFile(backup_file, "w", zipfile.ZIP_DEFLATED) as zip_file:
+            # Backup the database
+            self.stdout.write("Create database backup...")
+            with tempfile.TemporaryDirectory() as temp_dir:
+                backup_db_file = os.path.join(temp_dir, "db.sqlite3")
+                self.backup_database(backup_db_file)
+                zip_file.write(backup_db_file, "db.sqlite3")
+
+            # Backup the assets folder
+            if not os.path.exists(os.path.join("data", "assets")):
+                self.stdout.write(
+                    self.style.WARNING("No assets folder found. Skipping...")
+                )
+            else:
+                self.stdout.write("Backup bookmark assets...")
+                assets_folder = os.path.join("data", "assets")
+                for root, _, files in os.walk(assets_folder):
+                    for file in files:
+                        file_path = os.path.join(root, file)
+                        zip_file.write(file_path, os.path.join("assets", file))
+
+            # Backup the favicons folder
+            if not os.path.exists(os.path.join("data", "favicons")):
+                self.stdout.write(
+                    self.style.WARNING("No favicons folder found. Skipping...")
+                )
+            else:
+                self.stdout.write("Backup bookmark favicons...")
+                favicons_folder = os.path.join("data", "favicons")
+                for root, _, files in os.walk(favicons_folder):
+                    for file in files:
+                        file_path = os.path.join(root, file)
+                        zip_file.write(file_path, os.path.join("favicons", file))
+
+            # Backup the previews folder
+            if not os.path.exists(os.path.join("data", "previews")):
+                self.stdout.write(
+                    self.style.WARNING("No previews folder found. Skipping...")
+                )
+            else:
+                self.stdout.write("Backup bookmark previews...")
+                previews_folder = os.path.join("data", "previews")
+                for root, _, files in os.walk(previews_folder):
+                    for file in files:
+                        file_path = os.path.join(root, file)
+                        zip_file.write(file_path, os.path.join("previews", file))
+
+        self.stdout.write(self.style.SUCCESS(f"Backup created at {backup_file}"))
+
+    def backup_database(self, backup_db_file):
+        def progress(status, remaining, total):
+            self.stdout.write(f"Copied {total-remaining} of {total} pages...")
+
+        source_db = sqlite3.connect(os.path.join("data", "db.sqlite3"))
+        backup_db = sqlite3.connect(backup_db_file)
+        with backup_db:
+            source_db.backup(backup_db, pages=50, progress=progress)
+        backup_db.close()
+        source_db.close()

bookmarks/management/commands/migrate_tasks.py

@@ -0,0 +1,75 @@
+import json
+import os
+import sqlite3
+import importlib
+
+from django.core.management.base import BaseCommand
+
+
+class Command(BaseCommand):
+    help = "Migrate tasks from django-background-tasks to Huey"
+
+    def handle(self, *args, **options):
+        db = sqlite3.connect(os.path.join("data", "db.sqlite3"))
+
+        # Check if background_task table exists
+        cursor = db.cursor()
+        cursor.execute(
+            "SELECT name FROM sqlite_master WHERE type='table' AND name='background_task'"
+        )
+        row = cursor.fetchone()
+        if not row:
+            self.stdout.write(
+                "Legacy task table does not exist. Skipping task migration"
+            )
+            return
+
+        # Load legacy tasks
+        cursor.execute("SELECT id, task_name, task_params FROM background_task")
+        legacy_tasks = cursor.fetchall()
+
+        if len(legacy_tasks) == 0:
+            self.stdout.write("No legacy tasks found. Skipping task migration")
+            return
+
+        self.stdout.write(
+            f"Found {len(legacy_tasks)} legacy tasks. Migrating to Huey..."
+        )
+
+        # Migrate tasks to Huey
+        succeeded_tasks = []
+        for task in legacy_tasks:
+            task_id = task[0]
+            task_name = task[1]
+            task_params_json = task[2]
+            try:
+                task_params = json.loads(task_params_json)
+                function_params = task_params[0]
+
+                # Resolve task function
+                module_name, func_name = task_name.rsplit(".", 1)
+                module = importlib.import_module(module_name)
+                func = getattr(module, func_name)
+
+                # Call task function
+                func(*function_params)
+                succeeded_tasks.append(task_id)
+            except Exception:
+                self.stderr.write(f"Error migrating task [{task_id}] {task_name}")
+
+        self.stdout.write(f"Migrated {len(succeeded_tasks)} tasks successfully")
+
+        # Clean up
+        try:
+            placeholders = ", ".join("?" for _ in succeeded_tasks)
+            sql = f"DELETE FROM background_task WHERE id IN ({placeholders})"
+            cursor.execute(sql, succeeded_tasks)
+            db.commit()
+            self.stdout.write(
+                f"Deleted {len(succeeded_tasks)} migrated tasks from legacy table"
+            )
+        except Exception:
+            self.stderr.write("Error cleaning up legacy tasks")
+
+        cursor.close()
+        db.close()

bookmarks/migrations/0033_userprofile_default_mark_unread.py

@@ -0,0 +1,18 @@
+# Generated by Django 5.0.3 on 2024-04-17 19:27
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ("bookmarks", "0032_html_snapshots_hint_toast"),
+    ]
+
+    operations = [
+        migrations.AddField(
+            model_name="userprofile",
+            name="default_mark_unread",
+            field=models.BooleanField(default=False),
+        ),
+    ]

bookmarks/migrations/0034_bookmark_preview_image_file_and_more.py

@@ -0,0 +1,23 @@
+# Generated by Django 5.0.3 on 2024-05-10 07:01
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ("bookmarks", "0033_userprofile_default_mark_unread"),
+    ]
+
+    operations = [
+        migrations.AddField(
+            model_name="bookmark",
+            name="preview_image_file",
+            field=models.CharField(blank=True, max_length=512),
+        ),
+        migrations.AddField(
+            model_name="userprofile",
+            name="enable_preview_images",
+            field=models.BooleanField(default=False),
+        ),
+    ]

bookmarks/migrations/0035_userprofile_tag_grouping.py

@@ -0,0 +1,22 @@
+# Generated by Django 5.0.3 on 2024-05-14 08:28
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ("bookmarks", "0034_bookmark_preview_image_file_and_more"),
+    ]
+
+    operations = [
+        migrations.AddField(
+            model_name="userprofile",
+            name="tag_grouping",
+            field=models.CharField(
+                choices=[("alphabetical", "Alphabetical"), ("disabled", "Disabled")],
+                default="alphabetical",
+                max_length=12,
+            ),
+        ),
+    ]

bookmarks/migrations/0036_userprofile_auto_tagging_rules.py

@@ -0,0 +1,18 @@
+# Generated by Django 5.0.3 on 2024-05-17 07:09
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ("bookmarks", "0035_userprofile_tag_grouping"),
+    ]
+
+    operations = [
+        migrations.AddField(
+            model_name="userprofile",
+            name="auto_tagging_rules",
+            field=models.TextField(blank=True),
+        ),
+    ]

bookmarks/models.py

@@ -59,6 +59,7 @@ class Bookmark(models.Model):
     website_description = models.TextField(blank=True, null=True)
     web_archive_snapshot_url = models.CharField(max_length=2048, blank=True)
     favicon_file = models.CharField(max_length=512, blank=True)
+    preview_image_file = models.CharField(max_length=512, blank=True)
     unread = models.BooleanField(default=False)
     is_archived = models.BooleanField(default=False)
     shared = models.BooleanField(default=False)
@@ -91,6 +92,7 @@ class Bookmark(models.Model):
 
 class BookmarkAsset(models.Model):
     TYPE_SNAPSHOT = "snapshot"
+    TYPE_UPLOAD = "upload"
 
     CONTENT_TYPE_HTML = "text/html"
 
@@ -118,6 +120,9 @@ class BookmarkAsset(models.Model):
                 pass
         super().save(*args, **kwargs)
 
+    def __str__(self):
+        return self.display_name or f"Bookmark Asset #{self.pk}"
+
 
 @receiver(post_delete, sender=BookmarkAsset)
 def bookmark_asset_deleted(sender, instance, **kwargs):
@@ -347,6 +352,12 @@ class UserProfile(models.Model):
         (TAG_SEARCH_STRICT, "Strict"),
         (TAG_SEARCH_LAX, "Lax"),
     ]
+    TAG_GROUPING_ALPHABETICAL = "alphabetical"
+    TAG_GROUPING_DISABLED = "disabled"
+    TAG_GROUPING_CHOICES = [
+        (TAG_GROUPING_ALPHABETICAL, "Alphabetical"),
+        (TAG_GROUPING_DISABLED, "Disabled"),
+    ]
     user = models.OneToOneField(
         get_user_model(), related_name="profile", on_delete=models.CASCADE
     )
@@ -387,9 +398,16 @@ class UserProfile(models.Model):
         blank=False,
         default=TAG_SEARCH_STRICT,
     )
+    tag_grouping = models.CharField(
+        max_length=12,
+        choices=TAG_GROUPING_CHOICES,
+        blank=False,
+        default=TAG_GROUPING_ALPHABETICAL,
+    )
     enable_sharing = models.BooleanField(default=False, null=False)
     enable_public_sharing = models.BooleanField(default=False, null=False)
     enable_favicons = models.BooleanField(default=False, null=False)
+    enable_preview_images = models.BooleanField(default=False, null=False)
     display_url = models.BooleanField(default=False, null=False)
     display_view_bookmark_action = models.BooleanField(default=True, null=False)
     display_edit_bookmark_action = models.BooleanField(default=True, null=False)
@@ -397,8 +415,10 @@ class UserProfile(models.Model):
     display_remove_bookmark_action = models.BooleanField(default=True, null=False)
     permanent_notes = models.BooleanField(default=False, null=False)
     custom_css = models.TextField(blank=True, null=False)
+    auto_tagging_rules = models.TextField(blank=True, null=False)
     search_preferences = models.JSONField(default=dict, null=False)
     enable_automatic_html_snapshots = models.BooleanField(default=True, null=False)
+    default_mark_unread = models.BooleanField(default=False, null=False)
 
 
 class UserProfileForm(forms.ModelForm):
@@ -412,9 +432,11 @@ class UserProfileForm(forms.ModelForm):
             "bookmark_link_target",
             "web_archive_integration",
             "tag_search",
+            "tag_grouping",
             "enable_sharing",
             "enable_public_sharing",
             "enable_favicons",
+            "enable_preview_images",
             "enable_automatic_html_snapshots",
             "display_url",
             "display_view_bookmark_action",
@@ -422,7 +444,9 @@ class UserProfileForm(forms.ModelForm):
             "display_archive_bookmark_action",
             "display_remove_bookmark_action",
             "permanent_notes",
+            "default_mark_unread",
             "custom_css",
+            "auto_tagging_rules",
         ]
 
 

bookmarks/services/auto_tagging.py

@@ -0,0 +1,70 @@
+from urllib.parse import urlparse, parse_qs
+import re
+import idna
+
+
+def get_tags(script: str, url: str):
+    parsed_url = urlparse(url.lower())
+    result = set()
+
+    for line in script.lower().split("\n"):
+        if "#" in line:
+            i = line.index("#")
+            line = line[:i]
+
+        parts = line.split()
+        if len(parts) < 2:
+            continue
+
+        domain_pattern = re.sub("^https?://", "", parts[0])
+        path_pattern = None
+        qs_pattern = None
+
+        if "/" in domain_pattern:
+            i = domain_pattern.index("/")
+            path_pattern = domain_pattern[i:]
+            domain_pattern = domain_pattern[:i]
+
+        if path_pattern and "?" in path_pattern:
+            i = path_pattern.index("?")
+            qs_pattern = path_pattern[i + 1 :]
+            path_pattern = path_pattern[:i]
+
+        if not _domains_matches(domain_pattern, parsed_url.netloc):
+            continue
+
+        if path_pattern and not _path_matches(path_pattern, parsed_url.path):
+            continue
+
+        if qs_pattern and not _qs_matches(qs_pattern, parsed_url.query):
+            continue
+
+        for tag in parts[1:]:
+            result.add(tag)
+
+    return result
+
+
+def _path_matches(expected_path: str, actual_path: str) -> bool:
+    return actual_path.startswith(expected_path)
+
+
+def _domains_matches(expected_domain: str, actual_domain: str) -> bool:
+    expected_domain = idna.encode(expected_domain)
+    actual_domain = idna.encode(actual_domain)
+
+    return actual_domain.endswith(expected_domain)
+
+
+def _qs_matches(expected_qs: str, actual_qs: str) -> bool:
+    expected_qs = parse_qs(expected_qs, keep_blank_values=True)
+    actual_qs = parse_qs(actual_qs, keep_blank_values=True)
+
+    for key in expected_qs:
+        if key not in actual_qs:
+            return False
+        for value in expected_qs[key]:
+            if value != "" and value not in actual_qs[key]:
+                return False
+
+    return True

bookmarks/services/bookmarks.py

@@ -1,12 +1,19 @@
+import logging
+import os
 from typing import Union
 
+from django.conf import settings
 from django.contrib.auth.models import User
+from django.core.files.uploadedfile import UploadedFile
 from django.utils import timezone
 
-from bookmarks.models import Bookmark, parse_tag_string
-from bookmarks.services.tags import get_or_create_tags
-from bookmarks.services import website_loader
+from bookmarks.models import Bookmark, BookmarkAsset, parse_tag_string
 from bookmarks.services import tasks
+from bookmarks.services import website_loader
+from bookmarks.services import auto_tagging
+from bookmarks.services.tags import get_or_create_tags
+
+logger = logging.getLogger(__name__)
 
 
 def create_bookmark(bookmark: Bookmark, tag_string: str, current_user: User):
@@ -34,6 +41,8 @@ def create_bookmark(bookmark: Bookmark, tag_string: str, current_user: User):
     tasks.create_web_archive_snapshot(current_user, bookmark, False)
     # Load favicon
     tasks.load_favicon(current_user, bookmark)
+    # Load preview image
+    tasks.load_preview_image(current_user, bookmark)
     # Create HTML snapshot
     if current_user.profile.enable_automatic_html_snapshots:
         tasks.create_html_snapshot(bookmark)
@@ -52,6 +61,8 @@ def update_bookmark(bookmark: Bookmark, tag_string, current_user: User):
     bookmark.save()
     # Update favicon
     tasks.load_favicon(current_user, bookmark)
+    # Update preview image
+    tasks.load_preview_image(current_user, bookmark)
 
     if has_url_changed:
         # Update web archive snapshot, if URL changed
@@ -176,6 +187,46 @@ def unshare_bookmarks(bookmark_ids: [Union[int, str]], current_user: User):
     )
 
 
+def _generate_upload_asset_filename(asset: BookmarkAsset, filename: str):
+    formatted_datetime = asset.date_created.strftime("%Y-%m-%d_%H%M%S")
+    return f"{asset.asset_type}_{formatted_datetime}_{filename}"
+
+
+def upload_asset(bookmark: Bookmark, upload_file: UploadedFile) -> BookmarkAsset:
+    asset = BookmarkAsset(
+        bookmark=bookmark,
+        asset_type=BookmarkAsset.TYPE_UPLOAD,
+        content_type=upload_file.content_type,
+        display_name=upload_file.name,
+        status=BookmarkAsset.STATUS_PENDING,
+        gzip=False,
+    )
+    asset.save()
+
+    try:
+        filename = _generate_upload_asset_filename(asset, upload_file.name)
+        filepath = os.path.join(settings.LD_ASSET_FOLDER, filename)
+        with open(filepath, "wb") as f:
+            for chunk in upload_file.chunks():
+                f.write(chunk)
+        asset.status = BookmarkAsset.STATUS_COMPLETE
+        asset.file = filename
+        asset.file_size = upload_file.size
+        logger.info(
+            f"Successfully uploaded asset file. bookmark={bookmark} file={upload_file.name}"
+        )
+    except Exception as e:
+        logger.error(
+            f"Failed to upload asset file. bookmark={bookmark} file={upload_file.name}",
+            exc_info=e,
+        )
+        asset.status = BookmarkAsset.STATUS_FAILURE
+
+    asset.save()
+
+    return asset
+
+
 def _merge_bookmark_data(from_bookmark: Bookmark, to_bookmark: Bookmark):
     to_bookmark.title = from_bookmark.title
     to_bookmark.description = from_bookmark.description
@@ -192,6 +243,15 @@ def _update_website_metadata(bookmark: Bookmark):
 
 def _update_bookmark_tags(bookmark: Bookmark, tag_string: str, user: User):
     tag_names = parse_tag_string(tag_string)
+
+    if user.profile.auto_tagging_rules:
+        auto_tag_names = auto_tagging.get_tags(
+            user.profile.auto_tagging_rules, bookmark.url
+        )
+        for auto_tag_name in auto_tag_names:
+            if auto_tag_name not in tag_names:
+                tag_names.append(auto_tag_name)
+
     tags = get_or_create_tags(tag_names, user)
     bookmark.tags.set(tags)
 

bookmarks/services/importer.py

@@ -83,6 +83,8 @@ def import_netscape_html(
     tasks.schedule_bookmarks_without_snapshots(user)
     # Load favicons for newly imported bookmarks
     tasks.schedule_bookmarks_without_favicons(user)
+    # Load previews for newly imported bookmarks
+    tasks.schedule_bookmarks_without_previews(user)
 
     end = timezone.now()
     logger.debug(f"Import duration: {end - import_start}")

bookmarks/services/preview_image_loader.py

@@ -0,0 +1,88 @@
+import logging
+import mimetypes
+import os.path
+import hashlib
+from pathlib import Path
+
+import requests
+from django.conf import settings
+from bookmarks.services import website_loader
+
+logger = logging.getLogger(__name__)
+
+
+def _ensure_preview_folder():
+    Path(settings.LD_PREVIEW_FOLDER).mkdir(parents=True, exist_ok=True)
+
+
+def _url_to_filename(preview_image: str) -> str:
+    return hashlib.md5(preview_image.encode()).hexdigest()
+
+
+def _get_image_path(preview_image_file: str) -> Path:
+    return Path(os.path.join(settings.LD_PREVIEW_FOLDER, preview_image_file))
+
+
+def load_preview_image(url: str) -> str | None:
+    _ensure_preview_folder()
+
+    metadata = website_loader.load_website_metadata(url)
+    if not metadata.preview_image:
+        logger.debug(f"Could not find preview image in metadata: {url}")
+        return None
+
+    image_url = metadata.preview_image
+
+    logger.debug(f"Loading preview image: {image_url}")
+    with requests.get(image_url, stream=True) as response:
+        if response.status_code < 200 or response.status_code >= 300:
+            logger.debug(
+                f"Bad response status code for preview image: {image_url} status_code={response.status_code}"
+            )
+            return None
+
+        if "Content-Length" not in response.headers:
+            logger.debug(f"Empty Content-Length for preview image: {image_url}")
+            return None
+
+        content_length = int(response.headers["Content-Length"])
+        if content_length > settings.LD_PREVIEW_MAX_SIZE:
+            logger.debug(
+                f"Content-Length exceeds LD_PREVIEW_MAX_SIZE: {image_url} length={content_length}"
+            )
+            return None
+
+        if "Content-Type" not in response.headers:
+            logger.debug(f"Empty Content-Type for preview image: {image_url}")
+            return None
+
+        content_type = response.headers["Content-Type"].split(";", 1)[0]
+        file_extension = mimetypes.guess_extension(content_type)
+
+        if file_extension not in settings.LD_PREVIEW_ALLOWED_EXTENSIONS:
+            logger.debug(
+                f"Unsupported Content-Type for preview image: {image_url} content_type={content_type}"
+            )
+            return None
+
+        preview_image_hash = _url_to_filename(url)
+        preview_image_file = f"{preview_image_hash}{file_extension}"
+        preview_image_path = _get_image_path(preview_image_file)
+
+        with open(preview_image_path, "wb") as file:
+            downloaded = 0
+            for chunk in response.iter_content(chunk_size=8192):
+                downloaded += len(chunk)
+                if downloaded > content_length:
+                    logger.debug(
+                        f"Content-Length mismatch for preview image: {image_url} length={content_length} downloaded={downloaded}"
+                    )
+                    file.close()
+                    preview_image_path.unlink()
+                    return None
+
+                file.write(chunk)
+
+    logger.debug(f"Saved preview image as: {preview_image_path}")
+
+    return preview_image_file

bookmarks/services/singlefile.py

@@ -1,6 +1,9 @@
 import gzip
+import logging
 import os
+import shlex
 import shutil
+import signal
 import subprocess
 
 from django.conf import settings
@@ -10,16 +13,23 @@ class SingeFileError(Exception):
     pass
 
 
+logger = logging.getLogger(__name__)
+
+
 def create_snapshot(url: str, filepath: str):
     singlefile_path = settings.LD_SINGLEFILE_PATH
-    singlefile_options = settings.LD_SINGLEFILE_OPTIONS
+    # parse options to list of arguments
+    ublock_options = shlex.split(settings.LD_SINGLEFILE_UBLOCK_OPTIONS)
+    custom_options = shlex.split(settings.LD_SINGLEFILE_OPTIONS)
     temp_filepath = filepath + ".tmp"
-
+    # concat lists
+    args = [singlefile_path] + ublock_options + custom_options + [url, temp_filepath]
     try:
-        command = f"{singlefile_path} '{url}' {singlefile_options} {temp_filepath}"
-        subprocess.run(command, check=True, shell=True)
+        # Use start_new_session=True to create a new process group
+        process = subprocess.Popen(args, start_new_session=True)
+        process.wait(timeout=settings.LD_SINGLEFILE_TIMEOUT_SEC)
 
-        # single-file doesn't return exit codes apparently, so check if the file was created
+        # check if the file was created
         if not os.path.exists(temp_filepath):
             raise SingeFileError("Failed to create snapshot")
 
@@ -29,5 +39,20 @@ def create_snapshot(url: str, filepath: str):
             shutil.copyfileobj(raw_file, gz_file)
 
         os.remove(temp_filepath)
+    except subprocess.TimeoutExpired:
+        # First try to terminate properly
+        try:
+            logger.error(
+                "Timeout expired while creating snapshot. Terminating process..."
+            )
+            process.terminate()
+            process.wait(timeout=20)
+            raise SingeFileError("Timeout expired while creating snapshot")
+        except subprocess.TimeoutExpired:
+            # Kill the whole process group, which should also clean up any chromium
+            # processes spawned by single-file
+            logger.error("Timeout expired while terminating. Killing process...")
+            os.killpg(os.getpgid(process.pid), signal.SIGTERM)
+            raise SingeFileError("Timeout expired while creating snapshot")
     except subprocess.CalledProcessError as error:
         raise SingeFileError(f"Failed to create snapshot: {error.stderr}")

bookmarks/services/tasks.py

@@ -1,23 +1,56 @@
+import functools
 import logging
 import os
+from typing import List
 
 import waybackpy
-from background_task import background
-from background_task.models import Task
 from django.conf import settings
 from django.contrib.auth import get_user_model
 from django.contrib.auth.models import User
-from waybackpy.exceptions import WaybackError, TooManyRequestsError, NoCDXRecordFound
+from django.db.models import Q
 from django.utils import timezone, formats
+from huey import crontab
+from huey.contrib.djhuey import HUEY as huey
+from huey.exceptions import TaskLockedException
+from waybackpy.exceptions import WaybackError, TooManyRequestsError, NoCDXRecordFound
 
 import bookmarks.services.wayback
 from bookmarks.models import Bookmark, BookmarkAsset, UserProfile
-from bookmarks.services import favicon_loader, singlefile
+from bookmarks.services import favicon_loader, singlefile, preview_image_loader
 from bookmarks.services.website_loader import DEFAULT_USER_AGENT
 
 logger = logging.getLogger(__name__)
 
 
+# Create custom decorator for Huey tasks that implements exponential backoff
+# Taken from: https://huey.readthedocs.io/en/latest/guide.html#tips-and-tricks
+# Retry 1: 60
+# Retry 2: 240
+# Retry 3: 960
+# Retry 4: 3840
+# Retry 5: 15360
+def task(retries=5, retry_delay=15, retry_backoff=4):
+    def deco(fn):
+        @functools.wraps(fn)
+        def inner(*args, **kwargs):
+            task = kwargs.pop("task")
+            try:
+                return fn(*args, **kwargs)
+            except TaskLockedException as exc:
+                # Task locks are currently only used as workaround to enforce
+                # running specific types of tasks (e.g. singlefile snapshots)
+                # sequentially. In that case don't reduce the number of retries.
+                task.retries = retries
+                raise exc
+            except Exception as exc:
+                task.retry_delay *= retry_backoff
+                raise exc
+
+        return huey.task(retries=retries, retry_delay=retry_delay, context=True)(inner)
+
+    return deco
+
+
 def is_web_archive_integration_active(user: User) -> bool:
     background_tasks_enabled = not settings.LD_DISABLE_BACKGROUND_TASKS
     web_archive_integration_enabled = (
@@ -67,7 +100,7 @@ def _create_snapshot(bookmark: Bookmark):
     logger.info(f"Successfully created new snapshot for bookmark:. url={bookmark.url}")
 
 
-@background()
+@task()
 def _create_web_archive_snapshot_task(bookmark_id: int, force_update: bool):
     try:
         bookmark = Bookmark.objects.get(id=bookmark_id)
@@ -96,7 +129,7 @@ def _create_web_archive_snapshot_task(bookmark_id: int, force_update: bool):
     _load_newest_snapshot(bookmark)
 
 
-@background()
+@task()
 def _load_web_archive_snapshot_task(bookmark_id: int):
     try:
         bookmark = Bookmark.objects.get(id=bookmark_id)
@@ -114,13 +147,14 @@ def schedule_bookmarks_without_snapshots(user: User):
         _schedule_bookmarks_without_snapshots_task(user.id)
 
 
-@background()
+@task()
 def _schedule_bookmarks_without_snapshots_task(user_id: int):
     user = get_user_model().objects.get(id=user_id)
     bookmarks_without_snapshots = Bookmark.objects.filter(
         web_archive_snapshot_url__exact="", owner=user
     )
 
+    # TODO: Implement bulk task creation
     for bookmark in bookmarks_without_snapshots:
         # To prevent rate limit errors from the Wayback API only try to load the latest snapshots instead of creating
         # new ones when processing bookmarks in bulk
@@ -133,12 +167,18 @@ def is_favicon_feature_active(user: User) -> bool:
     return background_tasks_enabled and user.profile.enable_favicons
 
 
+def is_preview_feature_active(user: User) -> bool:
+    return (
+        user.profile.enable_preview_images and not settings.LD_DISABLE_BACKGROUND_TASKS
+    )
+
+
 def load_favicon(user: User, bookmark: Bookmark):
     if is_favicon_feature_active(user):
         _load_favicon_task(bookmark.id)
 
 
-@background()
+@task()
 def _load_favicon_task(bookmark_id: int):
     try:
         bookmark = Bookmark.objects.get(id=bookmark_id)
@@ -162,19 +202,15 @@ def schedule_bookmarks_without_favicons(user: User):
         _schedule_bookmarks_without_favicons_task(user.id)
 
 
-@background()
+@task()
 def _schedule_bookmarks_without_favicons_task(user_id: int):
     user = get_user_model().objects.get(id=user_id)
     bookmarks = Bookmark.objects.filter(favicon_file__exact="", owner=user)
-    tasks = []
 
+    # TODO: Implement bulk task creation
     for bookmark in bookmarks:
-        task = Task.objects.new_task(
-            task_name="bookmarks.services.tasks._load_favicon_task", args=(bookmark.id,)
-        )
-        tasks.append(task)
-
-    Task.objects.bulk_create(tasks)
+        _load_favicon_task(bookmark.id)
+        pass
 
 
 def schedule_refresh_favicons(user: User):
@@ -182,19 +218,59 @@ def schedule_refresh_favicons(user: User):
         _schedule_refresh_favicons_task(user.id)
 
 
-@background()
+@task()
 def _schedule_refresh_favicons_task(user_id: int):
     user = get_user_model().objects.get(id=user_id)
     bookmarks = Bookmark.objects.filter(owner=user)
-    tasks = []
 
+    # TODO: Implement bulk task creation
     for bookmark in bookmarks:
-        task = Task.objects.new_task(
-            task_name="bookmarks.services.tasks._load_favicon_task", args=(bookmark.id,)
-        )
-        tasks.append(task)
+        _load_favicon_task(bookmark.id)
 
-    Task.objects.bulk_create(tasks)
+
+def load_preview_image(user: User, bookmark: Bookmark):
+    if is_preview_feature_active(user):
+        _load_preview_image_task(bookmark.id)
+
+
+@task()
+def _load_preview_image_task(bookmark_id: int):
+    try:
+        bookmark = Bookmark.objects.get(id=bookmark_id)
+    except Bookmark.DoesNotExist:
+        return
+
+    logger.info(f"Load preview image for bookmark. url={bookmark.url}")
+
+    new_preview_image_file = preview_image_loader.load_preview_image(bookmark.url)
+
+    if new_preview_image_file != bookmark.preview_image_file:
+        bookmark.preview_image_file = new_preview_image_file or ""
+        bookmark.save(update_fields=["preview_image_file"])
+        logger.info(
+            f"Successfully updated preview image for bookmark. url={bookmark.url} preview_image_file={new_preview_image_file}"
+        )
+
+
+def schedule_bookmarks_without_previews(user: User):
+    if is_preview_feature_active(user):
+        _schedule_bookmarks_without_previews_task(user.id)
+
+
+@task()
+def _schedule_bookmarks_without_previews_task(user_id: int):
+    user = get_user_model().objects.get(id=user_id)
+    bookmarks = Bookmark.objects.filter(
+        Q(preview_image_file__exact=""),
+        owner=user,
+    )
+
+    # TODO: Implement bulk task creation
+    for bookmark in bookmarks:
+        try:
+            _load_preview_image_task(bookmark.id)
+        except Exception as exc:
+            logging.exception(exc)
 
 
 def is_html_snapshot_feature_active() -> bool:
@@ -205,6 +281,26 @@ def create_html_snapshot(bookmark: Bookmark):
     if not is_html_snapshot_feature_active():
         return
 
+    asset = _create_snapshot_asset(bookmark)
+    asset.save()
+
+
+def create_html_snapshots(bookmark_list: List[Bookmark]):
+    if not is_html_snapshot_feature_active():
+        return
+
+    assets_to_create = []
+    for bookmark in bookmark_list:
+        asset = _create_snapshot_asset(bookmark)
+        assets_to_create.append(asset)
+
+    BookmarkAsset.objects.bulk_create(assets_to_create)
+
+
+MAX_SNAPSHOT_FILENAME_LENGTH = 192
+
+
+def _create_snapshot_asset(bookmark: Bookmark) -> BookmarkAsset:
     timestamp = formats.date_format(timezone.now(), "SHORT_DATE_FORMAT")
     asset = BookmarkAsset(
         bookmark=bookmark,
@@ -213,8 +309,7 @@ def create_html_snapshot(bookmark: Bookmark):
         display_name=f"HTML snapshot from {timestamp}",
         status=BookmarkAsset.STATUS_PENDING,
     )
-    asset.save()
-    _create_html_snapshot_task(asset.id)
+    return asset
 
 
 def _generate_snapshot_filename(asset: BookmarkAsset) -> str:
@@ -227,10 +322,33 @@ def _generate_snapshot_filename(asset: BookmarkAsset) -> str:
     formatted_datetime = asset.date_created.strftime("%Y-%m-%d_%H%M%S")
     sanitized_url = "".join(sanitize_char(char) for char in asset.bookmark.url)
 
+    # Calculate the length of the non-URL parts of the filename
+    non_url_length = len(f"{asset.asset_type}{formatted_datetime}__.html.gz")
+    # Calculate the maximum length for the URL part
+    max_url_length = MAX_SNAPSHOT_FILENAME_LENGTH - non_url_length
+    # Truncate the URL if necessary
+    sanitized_url = sanitized_url[:max_url_length]
+
     return f"{asset.asset_type}_{formatted_datetime}_{sanitized_url}.html.gz"
 
 
-@background()
+# singe-file does not support running multiple instances in parallel, so we can
+# not queue up multiple snapshot tasks at once. Instead, schedule a periodic
+# task that grabs a number of pending assets and creates snapshots for them in
+# sequence. The task uses a lock to ensure that a new task isn't scheduled
+# before the previous one has finished.
+@huey.periodic_task(crontab(minute="*"))
+@huey.lock_task("schedule-html-snapshots-lock")
+def _schedule_html_snapshots_task():
+    # Get five pending assets
+    assets = BookmarkAsset.objects.filter(status=BookmarkAsset.STATUS_PENDING).order_by(
+        "date_created"
+    )[:5]
+
+    for asset in assets:
+        _create_html_snapshot_task(asset.id)
+
+
 def _create_html_snapshot_task(asset_id: int):
     try:
         asset = BookmarkAsset.objects.get(id=asset_id)
@@ -246,13 +364,34 @@ def _create_html_snapshot_task(asset_id: int):
         asset.status = BookmarkAsset.STATUS_COMPLETE
         asset.file = filename
         asset.gzip = True
+        asset.save()
         logger.info(
             f"Successfully created HTML snapshot for bookmark. url={asset.bookmark.url}"
         )
-    except singlefile.SingeFileError as error:
-        asset.status = BookmarkAsset.STATUS_FAILURE
+    except Exception as error:
         logger.error(
-            f"Failed to create HTML snapshot for bookmark. url={asset.bookmark.url}",
+            f"Failed to HTML snapshot for bookmark. url={asset.bookmark.url}",
             exc_info=error,
         )
-    asset.save()
+        asset.status = BookmarkAsset.STATUS_FAILURE
+        asset.save()
+
+
+def create_missing_html_snapshots(user: User) -> int:
+    if not is_html_snapshot_feature_active():
+        return 0
+
+    bookmarks_without_snapshots = Bookmark.objects.filter(owner=user).exclude(
+        bookmarkasset__asset_type=BookmarkAsset.TYPE_SNAPSHOT,
+        bookmarkasset__status__in=[
+            BookmarkAsset.STATUS_PENDING,
+            BookmarkAsset.STATUS_COMPLETE,
+        ],
+    )
+    bookmarks_without_snapshots |= Bookmark.objects.filter(owner=user).exclude(
+        bookmarkasset__asset_type=BookmarkAsset.TYPE_SNAPSHOT
+    )
+
+    create_html_snapshots(list(bookmarks_without_snapshots))
+
+    return bookmarks_without_snapshots.count()

bookmarks/services/website_loader.py

@@ -1,6 +1,7 @@
 import logging
 from dataclasses import dataclass
 from functools import lru_cache
+from urllib.parse import urljoin
 
 import requests
 from bs4 import BeautifulSoup
@@ -15,12 +16,14 @@ class WebsiteMetadata:
     url: str
     title: str
     description: str
+    preview_image: str | None
 
     def to_dict(self):
         return {
             "url": self.url,
             "title": self.title,
             "description": self.description,
+            "preview_image": self.preview_image,
         }
 
 
@@ -30,6 +33,7 @@ class WebsiteMetadata:
 def load_website_metadata(url: str):
     title = None
     description = None
+    preview_image = None
     try:
         start = timezone.now()
         page_text = load_page(url)
@@ -55,10 +59,21 @@ def load_website_metadata(url: str):
                 else None
             )
 
+        image_tag = soup.find("meta", attrs={"property": "og:image"})
+        preview_image = image_tag["content"].strip() if image_tag else None
+        if (
+            preview_image
+            and not preview_image.startswith("http://")
+            and not preview_image.startswith("https://")
+        ):
+            preview_image = urljoin(url, preview_image)
+
         end = timezone.now()
         logger.debug(f"Parsing duration: {end - start}")
     finally:
-        return WebsiteMetadata(url=url, title=title, description=description)
+        return WebsiteMetadata(
+            url=url, title=title, description=description, preview_image=preview_image
+        )
 
 
 CHUNK_SIZE = 50 * 1024

bookmarks/styles/bookmark-details.scss

@@ -33,6 +33,15 @@
     text-overflow: ellipsis;
   }
 
+  .preview-image {
+    margin: $unit-4 0;
+
+    img {
+      max-width: 100%;
+      max-height: 200px;
+    }
+  }
+
   dl {
     margin-bottom: 0;
   }
@@ -61,16 +70,22 @@
 
   .assets .asset-text {
     flex: 1 1 0;
+    gap: $unit-2;
+    min-width: 0;
+    display: flex;
+  }
+
+  .assets .asset-text .truncate {
+    flex-shrink: 1;
   }
 
   .assets .asset-text .filesize {
     color: $gray-color;
-    margin-left: $unit-2;
   }
 
   .assets .asset-actions, .assets-actions {
     display: flex;
-    gap: $unit-3;
+    gap: $unit-4;
     align-items: center;
   }
 

bookmarks/styles/bookmark-form.scss

@@ -36,12 +36,11 @@
   .form-input-hint.bookmark-exists {
     display: none;
     color: $warning-color;
+  }
 
-    a {
-      color: $warning-color;
-      text-decoration: underline;
-      font-weight: bold;
-    }
+  .form-input-hint.auto-tags {
+    display: none;
+    color: $success-color;
   }
 
   details.notes textarea {

bookmarks/styles/bookmark-page.scss

@@ -128,9 +128,26 @@ ul.bookmark-list {
 /* Bookmarks */
 li[ld-bookmark-item] {
   position: relative;
+  display: flex;
+  gap: $unit-2;
   margin-top: $unit-2;
 
-  [ld-bulk-edit-checkbox].form-checkbox {
+  .content {
+    flex: 1 1 0;
+    min-width: 0;
+  }
+
+  img.preview-image {
+    flex: 0 0 auto;
+    width: 100px;
+    height: 60px;
+    margin-top: $unit-h;
+    object-fit: cover;
+    border-radius: $border-radius;
+    border: solid 1px $border-color-dark;
+  }
+
+  .form-checkbox.bulk-edit-checkbox {
     display: none;
   }
 
@@ -182,6 +199,12 @@ li[ld-bookmark-item] {
     animation: 0.3s ease 0s appear;
   }
 
+  @media (pointer:coarse) {
+    .title a[data-tooltip]::after {
+      display: none;
+    }
+  }
+
   &.unread .title a {
     font-style: italic;
   }
@@ -323,7 +346,7 @@ $bulk-edit-transition-duration: 400ms;
   }
 
   /* All checkbox */
-  [ld-bulk-edit-checkbox][all].form-checkbox {
+  .form-checkbox.bulk-edit-checkbox.all {
     display: block;
     width: $bulk-edit-toggle-width;
     margin: 0 0 0 $bulk-edit-toggle-offset;
@@ -331,7 +354,7 @@ $bulk-edit-transition-duration: 400ms;
   }
 
   /* Bookmark checkboxes */
-  li[ld-bookmark-item] [ld-bulk-edit-checkbox].form-checkbox {
+  li[ld-bookmark-item] .form-checkbox.bulk-edit-checkbox {
     display: block;
     position: absolute;
     width: $bulk-edit-toggle-width;
@@ -346,11 +369,11 @@ $bulk-edit-transition-duration: 400ms;
     transition: all $bulk-edit-transition-duration;
 
     .form-icon {
-        top: 0;
+      top: 0;
     }
   }
 
-  &.active li[ld-bookmark-item] [ld-bulk-edit-checkbox].form-checkbox {
+  &.active li[ld-bookmark-item] .form-checkbox.bulk-edit-checkbox {
     visibility: visible;
     opacity: 1;
   }

bookmarks/styles/reader-mode.scss

@@ -0,0 +1,27 @@
+html.reader-mode {
+  --font-size: 1rem;
+  line-height: 1.6;
+
+  body {
+    margin: 3rem 2rem;
+  }
+
+  .container {
+    max-width: 600px;
+  }
+
+  .byline {
+    font-style: italic;
+    font-size: 0.8rem;
+  }
+
+  .reading-time {
+    font-size: 0.7rem;
+  }
+
+  img {
+    max-width: 100%;
+    height: auto;
+  }
+}
+

bookmarks/styles/theme-dark.scss

@@ -12,6 +12,7 @@
 @import "bookmark-form";
 @import "settings";
 @import "markdown";
+@import "reader-mode";
 
 /* Dark theme overrides */
 

bookmarks/styles/theme-light.scss

@@ -12,3 +12,4 @@
 @import "bookmark-form";
 @import "settings";
 @import "markdown";
+@import "reader-mode";

bookmarks/tasks.py

@@ -0,0 +1,2 @@
+# Expose task modules to Huey Django extension
+import bookmarks.services.tasks

bookmarks/templates/admin/background_tasks.html

@@ -0,0 +1,39 @@
+{% extends "admin/base_site.html" %}
+
+{% block content %}
+  <table style="width: 100%">
+    <thead>
+    <tr>
+      <th>ID</th>
+      <th>Name</th>
+      <th>Args</th>
+      <th>Retries</th>
+    </tr>
+    </thead>
+    <tbody>
+    {% for task in tasks %}
+      <tr>
+        <td>{{ task.id }}</td>
+        <td>{{ task.name }}</td>
+        <td>{{ task.args }}</td>
+        <td>{{ task.retries }}</td>
+      </tr>
+    {% endfor %}
+    </tbody>
+  </table>
+  <p class="paginator">
+    {% if page.paginator.num_pages > 1 %}
+      {% for page_number in page_range %}
+        {% if page_number == page.number %}
+          <span class="this-page">{{ page_number }}</span>
+        {% elif page_number == '…' %}
+          <span>…</span>
+        {% else %}
+          <a href="?p={{ page_number }}">{{ page_number }}</a>
+        {% endif %}
+      {% endfor %}
+      &nbsp;
+    {% endif %}
+    {{ page.paginator.count }} tasks
+  </p>
+{% endblock %}

bookmarks/templates/bookmarks/archive.html

@@ -4,11 +4,7 @@
 {% load bookmarks %}
 
 {% block content %}
-  <div class="bookmarks-page grid columns-md-1"
-       ld-bulk-edit
-       ld-bookmark-page
-       bookmarks-url="{% url 'bookmarks:partials.bookmark_list.archived' %}"
-       tags-url="{% url 'bookmarks:partials.tag_cloud.archived' %}">
+  <div ld-bulk-edit class="bookmarks-page grid columns-md-1">
 
     {# Bookmark list #}
     <section class="content-area col-2">
@@ -17,17 +13,22 @@
         <div class="header-controls">
           {% bookmark_search bookmark_list.search tag_cloud.tags mode='archived' %}
           {% include 'bookmarks/bulk_edit/toggle.html' %}
-          <button ld-modal modal-content=".tag-cloud" class="btn ml-2 show-md">Tags</button>
+          <button ld-fetch="{{ bookmark_list.tag_modal_url }}" ld-target="body|append" ld-on="click"
+                  class="btn ml-2 show-md">Tags
+          </button>
         </div>
       </div>
 
-      <form class="bookmark-actions"
+      <form ld-form ld-fire="refresh-bookmark-list,refresh-tag-cloud"
+            class="bookmark-actions"
             action="{{ bookmark_list.action_url|safe }}"
             method="post" autocomplete="off">
         {% csrf_token %}
         {% include 'bookmarks/bulk_edit/bar.html' with disable_actions='bulk_archive' %}
 
-        <div class="bookmark-list-container">
+        <div ld-fetch="{{ bookmark_list.refresh_url }}" ld-on="refresh-bookmark-list"
+             ld-fire="refresh-bookmark-list-done"
+             class="bookmark-list-container">
           {% include 'bookmarks/bookmark_list.html' %}
         </div>
       </form>
@@ -38,7 +39,8 @@
       <div class="content-area-header">
         <h2>Tags</h2>
       </div>
-      <div class="tag-cloud-container">
+      <div ld-fetch="{{ tag_cloud.refresh_url }}" ld-on="refresh-tag-cloud"
+           class="tag-cloud-container">
         {% include 'bookmarks/tag_cloud.html' %}
       </div>
     </section>

bookmarks/templates/bookmarks/bookmark_list.html

@@ -10,142 +10,143 @@
       data-bookmarks-total="{{ bookmark_list.bookmarks_total }}">
     {% for bookmark_item in bookmark_list.items %}
       <li ld-bookmark-item{% if bookmark_item.css_classes %} class="{{ bookmark_item.css_classes }}"{% endif %}>
-        <div class="title">
-          <label ld-bulk-edit-checkbox class="form-checkbox">
-            <input type="checkbox" name="bookmark_id" value="{{ bookmark_item.id }}">
-            <i class="form-icon"></i>
-          </label>
-          {% if bookmark_item.favicon_file and bookmark_list.show_favicons %}
-            <img src="{% static bookmark_item.favicon_file %}" alt="">
-          {% endif %}
-          <a href="{{ bookmark_item.url }}" target="{{ bookmark_list.link_target }}" rel="noopener">
-            <span>{{ bookmark_item.title }}</span>
-          </a>
-        </div>
-        {% if bookmark_list.show_url %}
-          <div class="url-path truncate">
-            <a href="{{ bookmark_item.url }}" target="{{ bookmark_list.link_target }}" rel="noopener"
-               class="url-display">
-              {{ bookmark_item.url }}
+        <div class="content">
+          <div class="title">
+            <label class="form-checkbox bulk-edit-checkbox">
+              <input type="checkbox" name="bookmark_id" value="{{ bookmark_item.id }}">
+              <i class="form-icon"></i>
+            </label>
+            {% if bookmark_item.favicon_file and bookmark_list.show_favicons %}
+              <img class="favicon" src="{% static bookmark_item.favicon_file %}" alt="">
+            {% endif %}
+            <a href="{{ bookmark_item.url }}" target="{{ bookmark_list.link_target }}" rel="noopener">
+              <span>{{ bookmark_item.title }}</span>
             </a>
           </div>
-        {% endif %}
-        {% if bookmark_list.description_display == 'inline' %}
-          <div class="description inline truncate">
+          {% if bookmark_list.show_url %}
+            <div class="url-path truncate">
+              <a href="{{ bookmark_item.url }}" target="{{ bookmark_list.link_target }}" rel="noopener"
+                class="url-display">
+                {{ bookmark_item.url }}
+              </a>
+            </div>
+          {% endif %}
+          {% if bookmark_list.description_display == 'inline' %}
+            <div class="description inline truncate">
+              {% if bookmark_item.tag_names %}
+                <span class="tags">
+                  {% for tag_name in bookmark_item.tag_names %}
+                    <a href="?{% add_tag_to_query tag_name %}">{{ tag_name|hash_tag }}</a>
+                  {% endfor %}
+                </span>
+              {% endif %}
+              {% if bookmark_item.tag_names and bookmark_item.description %} | {% endif %}
+              {% if bookmark_item.description %}
+                <span>{{ bookmark_item.description }}</span>
+              {% endif %}
+            </div>
+          {% else %}
+            {% if bookmark_item.description %}
+              <div class="description separate">{{ bookmark_item.description }}</div>
+            {% endif %}
             {% if bookmark_item.tag_names %}
-              <span class="tags">
+              <div class="tags">
                 {% for tag_name in bookmark_item.tag_names %}
                   <a href="?{% add_tag_to_query tag_name %}">{{ tag_name|hash_tag }}</a>
                 {% endfor %}
+              </div>
+            {% endif %}
+          {% endif %}
+          {% if bookmark_item.notes %}
+            <div class="notes bg-gray text-gray-dark">
+              <div class="markdown">{% markdown bookmark_item.notes %}</div>
+            </div>
+          {% endif %}
+          <div class="actions text-gray">
+            {% if bookmark_item.display_date %}
+              {% if bookmark_item.web_archive_snapshot_url %}
+                <a href="{{ bookmark_item.web_archive_snapshot_url }}"
+                  title="Show snapshot on the Internet Archive Wayback Machine"
+                  target="{{ bookmark_list.link_target }}"
+                  rel="noopener">
+                  {{ bookmark_item.display_date }} ∞
+                </a>
+              {% else %}
+                <span>{{ bookmark_item.display_date }}</span>
+              {% endif %}
+              <span>|</span>
+            {% endif %}
+            {# View link is visible for both owned and shared bookmarks #}
+            {% if bookmark_list.show_view_action %}
+              <a ld-fetch="{% url 'bookmarks:details_modal' bookmark_item.id %}?return_url={{ bookmark_list.return_url|urlencode }}"
+                ld-on="click" ld-target="body|append"
+                href="{% url 'bookmarks:details' bookmark_item.id %}">View</a>
+            {% endif %}
+            {% if bookmark_item.is_editable %}
+              {# Bookmark owner actions #}
+              {% if bookmark_list.show_edit_action %}
+                <a href="{% url 'bookmarks:edit' bookmark_item.id %}?return_url={{ bookmark_list.return_url|urlencode }}">Edit</a>
+              {% endif %}
+              {% if bookmark_list.show_archive_action %}
+                {% if bookmark_item.is_archived %}
+                  <button type="submit" name="unarchive" value="{{ bookmark_item.id }}"
+                          class="btn btn-link btn-sm">Unarchive
+                  </button>
+                {% else %}
+                  <button type="submit" name="archive" value="{{ bookmark_item.id }}"
+                          class="btn btn-link btn-sm">Archive
+                  </button>
+                {% endif %}
+              {% endif %}
+              {% if bookmark_list.show_remove_action %}
+                <button ld-confirm-button type="submit" name="remove" value="{{ bookmark_item.id }}"
+                        class="btn btn-link btn-sm">Remove
+                </button>
+              {% endif %}
+            {% else %}
+              {# Shared bookmark actions #}
+              <span>Shared by
+                <a href="?{% replace_query_param user=bookmark_item.owner.username %}">{{ bookmark_item.owner.username }}</a>
               </span>
             {% endif %}
-            {% if bookmark_item.tag_names and bookmark_item.description %} | {% endif %}
-            {% if bookmark_item.description %}
-              <span>{{ bookmark_item.description }}</span>
+            {% if bookmark_item.has_extra_actions %}
+              <div class="extra-actions">
+                <span class="hide-sm">|</span>
+                {% if bookmark_item.show_mark_as_read %}
+                  <button type="submit" name="mark_as_read" value="{{ bookmark_item.id }}"
+                          class="btn btn-link btn-sm btn-icon"
+                          ld-confirm-button ld-confirm-icon="ld-icon-read" ld-confirm-question="Mark as read?">
+                    <svg xmlns="http://www.w3.org/2000/svg" width="16" height="16">
+                      <use xlink:href="#ld-icon-unread"></use>
+                    </svg>
+                    Unread
+                  </button>
+                {% endif %}
+                {% if bookmark_item.show_unshare %}
+                  <button type="submit" name="unshare" value="{{ bookmark_item.id }}"
+                          class="btn btn-link btn-sm btn-icon"
+                          ld-confirm-button ld-confirm-icon="ld-icon-unshare" ld-confirm-question="Unshare?">
+                    <svg xmlns="http://www.w3.org/2000/svg" width="16" height="16">
+                      <use xlink:href="#ld-icon-share"></use>
+                    </svg>
+                    Shared
+                  </button>
+                {% endif %}
+                {% if bookmark_item.show_notes_button %}
+                  <button type="button" class="btn btn-link btn-sm btn-icon toggle-notes">
+                    <svg xmlns="http://www.w3.org/2000/svg" width="16" height="16">
+                      <use xlink:href="#ld-icon-note"></use>
+                    </svg>
+                    Notes
+                  </button>
+                {% endif %}
+              </div>
             {% endif %}
           </div>
-        {% else %}
-          {% if bookmark_item.description %}
-            <div class="description separate">
-              {{ bookmark_item.description }}
-            </div>
-          {% endif %}
-          {% if bookmark_item.tag_names %}
-            <div class="tags">
-              {% for tag_name in bookmark_item.tag_names %}
-                <a href="?{% add_tag_to_query tag_name %}">{{ tag_name|hash_tag }}</a>
-              {% endfor %}
-            </div>
-          {% endif %}
-        {% endif %}
-        {% if bookmark_item.notes %}
-          <div class="notes bg-gray text-gray-dark">
-            <div class="markdown">
-              {% markdown bookmark_item.notes %}
-            </div>
-          </div>
-        {% endif %}
-        <div class="actions text-gray">
-          {% if bookmark_item.display_date %}
-            {% if bookmark_item.web_archive_snapshot_url %}
-              <a href="{{ bookmark_item.web_archive_snapshot_url }}"
-                 title="Show snapshot on the Internet Archive Wayback Machine"
-                 target="{{ bookmark_list.link_target }}"
-                 rel="noopener">
-                {{ bookmark_item.display_date }} ∞
-              </a>
-            {% else %}
-              <span>{{ bookmark_item.display_date }}</span>
-            {% endif %}
-            <span>|</span>
-          {% endif %}
-          {# View link is visible for both owned and shared bookmarks #}
-          {% if bookmark_list.show_view_action %}
-            <a ld-modal
-               modal-url="{% url 'bookmarks:details_modal' bookmark_item.id %}?return_url={{ bookmark_list.return_url|urlencode }}"
-               href="{% url 'bookmarks:details' bookmark_item.id %}">View</a>
-          {% endif %}
-          {% if bookmark_item.is_editable %}
-            {# Bookmark owner actions #}
-            {% if bookmark_list.show_edit_action %}
-              <a href="{% url 'bookmarks:edit' bookmark_item.id %}?return_url={{ bookmark_list.return_url|urlencode }}">Edit</a>
-            {% endif %}
-            {% if bookmark_list.show_archive_action %}
-              {% if bookmark_item.is_archived %}
-                <button type="submit" name="unarchive" value="{{ bookmark_item.id }}"
-                        class="btn btn-link btn-sm">Unarchive
-                </button>
-              {% else %}
-                <button type="submit" name="archive" value="{{ bookmark_item.id }}"
-                        class="btn btn-link btn-sm">Archive
-                </button>
-              {% endif %}
-            {% endif %}
-            {% if bookmark_list.show_remove_action %}
-              <button ld-confirm-button type="submit" name="remove" value="{{ bookmark_item.id }}"
-                      class="btn btn-link btn-sm">Remove
-              </button>
-            {% endif %}
-          {% else %}
-            {# Shared bookmark actions #}
-            <span>Shared by
-              <a href="?{% replace_query_param user=bookmark_item.owner.username %}">{{ bookmark_item.owner.username }}</a>
-            </span>
-          {% endif %}
-          {% if bookmark_item.has_extra_actions %}
-            <div class="extra-actions">
-              <span class="hide-sm">|</span>
-              {% if bookmark_item.show_mark_as_read %}
-                <button type="submit" name="mark_as_read" value="{{ bookmark_item.id }}"
-                        class="btn btn-link btn-sm btn-icon"
-                        ld-confirm-button confirm-icon="ld-icon-read" confirm-question="Mark as read?">
-                  <svg xmlns="http://www.w3.org/2000/svg" width="16" height="16">
-                    <use xlink:href="#ld-icon-unread"></use>
-                  </svg>
-                  Unread
-                </button>
-              {% endif %}
-              {% if bookmark_item.show_unshare %}
-                <button type="submit" name="unshare" value="{{ bookmark_item.id }}"
-                        class="btn btn-link btn-sm btn-icon"
-                        ld-confirm-button confirm-icon="ld-icon-unshare" confirm-question="Unshare?">
-                  <svg xmlns="http://www.w3.org/2000/svg" width="16" height="16">
-                    <use xlink:href="#ld-icon-share"></use>
-                  </svg>
-                  Shared
-                </button>
-              {% endif %}
-              {% if bookmark_item.show_notes_button %}
-                <button type="button" class="btn btn-link btn-sm btn-icon toggle-notes">
-                  <svg xmlns="http://www.w3.org/2000/svg" width="16" height="16">
-                    <use xlink:href="#ld-icon-note"></use>
-                  </svg>
-                  Notes
-                </button>
-              {% endif %}
-            </div>
-          {% endif %}
         </div>
+        {% if bookmark_list.show_preview_images and bookmark_item.preview_image_file %}
+          <img class="preview-image" src="{% static bookmark_item.preview_image_file %}" loading="lazy"/>
+        {% endif %}
       </li>
     {% endfor %}
   </ul>

bookmarks/templates/bookmarks/bulk_edit/bar.html

@@ -2,7 +2,7 @@
 {% htmlmin %}
   <div class="bulk-edit-bar">
     <div class="bulk-edit-actions bg-gray">
-      <label ld-bulk-edit-checkbox all class="form-checkbox">
+      <label class="form-checkbox bulk-edit-checkbox all">
         <input type="checkbox">
         <i class="form-icon"></i>
       </label>

bookmarks/templates/bookmarks/bulk_edit/toggle.html

@@ -1,4 +1,4 @@
-<button ld-bulk-edit-active-toggle class="btn hide-sm ml-2" title="Bulk edit">
+<button class="btn hide-sm ml-2 bulk-edit-active-toggle" title="Bulk edit">
   <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 20 20" fill="currentColor" width="20px"
        height="20px">
     <path

bookmarks/templates/bookmarks/details/assets.html

@@ -1,37 +1,48 @@
-{% if details.assets %}
-  <div class="assets">
-    {% for asset in details.assets %}
-      <div class="asset" data-asset-id="{{ asset.id }}">
-        <div class="asset-icon {{ asset.icon_classes }}">
-          {% include 'bookmarks/details/asset_icon.html' %}
+<div {% if details.has_pending_assets %}
+  ld-fetch="{% url 'bookmarks:details_assets' details.bookmark.id %}"
+  ld-interval="5" ld-target="self|outerHTML"
+{% endif %}>
+  {% if details.assets %}
+    <div class="assets">
+      {% for asset in details.assets %}
+        <div class="asset" data-asset-id="{{ asset.id }}">
+          <div class="asset-icon {{ asset.icon_classes }}">
+            {% include 'bookmarks/details/asset_icon.html' %}
+          </div>
+          <div class="asset-text {{ asset.text_classes }}">
+          <span class="truncate">
+            {{ asset.display_name }}
+            {% if asset.status == 'pending' %}(queued){% endif %}
+            {% if asset.status == 'failure' %}(failed){% endif %}
+          </span>
+            {% if asset.file_size %}
+              <span class="filesize">{{ asset.file_size|filesizeformat }}</span>
+            {% endif %}
+          </div>
+          <div class="asset-actions">
+            {% if asset.file %}
+              <a class="btn btn-link" href="{% url 'bookmarks:assets.view' asset.id %}" target="_blank">View</a>
+            {% endif %}
+            {% if details.is_editable %}
+              <button ld-confirm-button type="submit" name="remove_asset" value="{{ asset.id }}" class="btn btn-link">
+                Remove
+              </button>
+            {% endif %}
+          </div>
         </div>
-        <div class="asset-text truncate {{ asset.text_classes }}">
-        <span>
-          {{ asset.display_name }}
-          {% if asset.status == 'pending' %}(queued){% endif %}
-          {% if asset.status == 'failure' %}(failed){% endif %}
-        </span>
-          {% if asset.file_size %}
-            <span class="filesize">{{ asset.file_size|filesizeformat }}</span>
-          {% endif %}
-        </div>
-        <div class="asset-actions">
-          {% if asset.file %}
-            <a class="btn btn-link" href="{% url 'bookmarks:assets.view' asset.id %}" target="_blank">View</a>
-          {% endif %}
-          {% if details.is_editable %}
-            <button ld-confirm-button type="submit" name="remove_asset" value="{{ asset.id }}" class="btn btn-link">
-              Remove
-            </button>
-          {% endif %}
-        </div>
-      </div>
-    {% endfor %}
-  </div>
-{% endif %}
+      {% endfor %}
+    </div>
+  {% endif %}
 
-{% if details.is_editable %}
-  <div class="assets-actions">
-    <button type="submit" name="create_snapshot" class="btn btn-link">Create HTML snapshot</button>
-  </div>
-{% endif %}
+  {% if details.is_editable %}
+    <div class="assets-actions">
+      <button type="submit" name="create_snapshot" class="btn btn-link"
+              {% if details.has_pending_assets %}disabled{% endif %}>Create HTML snapshot
+      </button>
+      <button ld-upload-button id="upload-asset" name="upload_asset" value="{{ details.bookmark.id }}" type="button"
+              class="btn btn-link">Upload file
+      </button>
+      <input id="upload-asset-file" name="upload_asset_file" type="file" class="d-hide">
+    </div>
+  {% endif %}
+</div>

bookmarks/templates/bookmarks/details/form.html

@@ -1,9 +1,8 @@
 {% load static %}
 {% load shared %}
 
-<form ld-form action="{% url 'bookmarks:details' details.bookmark.id %}"
-      refresh-url="{% url 'bookmarks:partials.details_form' details.bookmark.id %}"
-      refresh-events="bookmark-page-refresh"
+<form ld-form ld-fire="refresh-bookmark-list,refresh-tag-cloud,refresh-details"
+      action="{% url 'bookmarks:details' details.bookmark.id %}"
       method="post">
   <div class="weblinks">
     <a class="weblink" href="{{ details.bookmark.url }}" rel="noopener"
@@ -13,6 +12,17 @@
       {% endif %}
       <span>{{ details.bookmark.url }}</span>
     </a>
+    {% if details.latest_snapshot %}
+      <a class="weblink" href="{% url 'bookmarks:assets.read' details.latest_snapshot.id %}"
+         target="{{ details.profile.bookmark_link_target }}">
+        {% if details.show_link_icons %}
+          <svg class="favicon" xmlns="http://www.w3.org/2000/svg">
+            <use xlink:href="#ld-icon-unread"></use>
+          </svg>
+        {% endif %}
+        <span>Reader mode</span>
+      </a>
+    {% endif %}
     {% if details.bookmark.web_archive_snapshot_url %}
       <a class="weblink" href="{{ details.bookmark.web_archive_snapshot_url }}"
          target="{{ details.profile.bookmark_link_target }}">
@@ -23,10 +33,15 @@
                 fill="currentColor" fill-rule="evenodd"/>
           </svg>
         {% endif %}
-        <span>View on Internet Archive</span>
+        <span>Internet Archive</span>
       </a>
     {% endif %}
   </div>
+  {% if details.preview_image_enabled and details.bookmark.preview_image_file %}
+    <div class="preview-image">
+      <img src="{% static details.bookmark.preview_image_file %}"/>
+    </div>
+  {% endif %}
   <dl class="grid columns-2 columns-sm-1 gap-0">
     {% if details.is_editable %}
       <div class="status col-2">
@@ -35,14 +50,14 @@
           {% csrf_token %}
           <div class="form-group">
             <label class="form-switch">
-              <input ld-form-auto-submit type="checkbox" name="is_archived"
+              <input ld-auto-submit type="checkbox" name="is_archived"
                      {% if details.bookmark.is_archived %}checked{% endif %}>
               <i class="form-icon"></i> Archived
             </label>
           </div>
           <div class="form-group">
             <label class="form-switch">
-              <input ld-form-auto-submit type="checkbox" name="unread"
+              <input ld-auto-submit type="checkbox" name="unread"
                      {% if details.bookmark.unread %}checked{% endif %}>
               <i class="form-icon"></i> Unread
             </label>
@@ -50,7 +65,7 @@
           {% if details.profile.enable_sharing %}
             <div class="form-group">
               <label class="form-switch">
-                <input ld-form-auto-submit type="checkbox" name="shared"
+                <input ld-auto-submit type="checkbox" name="shared"
                        {% if details.bookmark.shared %}checked{% endif %}>
                 <i class="form-icon"></i> Shared
               </label>

bookmarks/templates/bookmarks/details_modal.html

@@ -1,4 +1,7 @@
-<div class="modal active bookmark-details">
+<div ld-modal
+     ld-fetch="{% url 'bookmarks:details_modal' details.bookmark.id %}" ld-on="refresh-details"
+     ld-select=".content" ld-target=".modal.bookmark-details .content|outerHTML"
+     class="modal active bookmark-details">
   <div class="modal-overlay" aria-label="Close"></div>
   <div class="modal-container">
     <div class="modal-header">

bookmarks/templates/bookmarks/form.html

@@ -23,10 +23,10 @@
     <label for="{{ form.tag_string.id_for_label }}" class="form-label">Tags</label>
     {{ form.tag_string|add_class:"form-input"|attr:"ld-tag-autocomplete"|attr:"autocomplete:off"|attr:"autocapitalize:off" }}
     <div class="form-input-hint">
-      Enter any number of tags separated by space and <strong>without</strong> the hash (#). If a tag does not
-      exist it will be
-      automatically created.
+      Enter any number of tags separated by space and <strong>without</strong> the hash (#).
+      If a tag does not exist it will be automatically created.
     </div>
+    <div class="form-input-hint auto-tags"></div>
     {{ form.tag_string.errors }}
   </div>
   <div class="form-group has-icon-right">
@@ -197,6 +197,18 @@
             } else {
               bookmarkExistsHint.style['display'] = 'none';
             }
+
+            // Preview auto tags
+            const autoTags = data.auto_tags;
+            const autoTagsHint = document.querySelector('.form-input-hint.auto-tags');
+
+            if (autoTags.length > 0) {
+              autoTags.sort();
+              autoTagsHint.style['display'] = 'block';
+              autoTagsHint.innerHTML = `Auto tags: ${autoTags.join(" ")}`;
+            } else {
+              autoTagsHint.style['display'] = 'none';
+            }
           });
       }
 

bookmarks/templates/bookmarks/index.html

@@ -4,11 +4,7 @@
 {% load bookmarks %}
 
 {% block content %}
-  <div class="bookmarks-page grid columns-md-1"
-       ld-bulk-edit
-       ld-bookmark-page
-       bookmarks-url="{% url 'bookmarks:partials.bookmark_list.active' %}"
-       tags-url="{% url 'bookmarks:partials.tag_cloud.active' %}">
+  <div ld-bulk-edit class="bookmarks-page grid columns-md-1">
 
     {# Bookmark list #}
     <section class="content-area col-2">
@@ -17,17 +13,22 @@
         <div class="header-controls">
           {% bookmark_search bookmark_list.search tag_cloud.tags %}
           {% include 'bookmarks/bulk_edit/toggle.html' %}
-          <button ld-modal modal-content=".tag-cloud" class="btn ml-2 show-md">Tags</button>
+          <button ld-fetch="{{ bookmark_list.tag_modal_url }}" ld-target="body|append" ld-on="click"
+                  class="btn ml-2 show-md">Tags
+          </button>
         </div>
       </div>
 
-      <form class="bookmark-actions"
+      <form ld-form ld-fire="refresh-bookmark-list,refresh-tag-cloud"
+            class="bookmark-actions"
             action="{{ bookmark_list.action_url|safe }}"
             method="post" autocomplete="off">
         {% csrf_token %}
         {% include 'bookmarks/bulk_edit/bar.html' with disable_actions='bulk_unarchive' %}
 
-        <div class="bookmark-list-container">
+        <div ld-fetch="{{ bookmark_list.refresh_url }}" ld-on="refresh-bookmark-list"
+             ld-fire="refresh-bookmark-list-done"
+             class="bookmark-list-container">
           {% include 'bookmarks/bookmark_list.html' %}
         </div>
       </form>
@@ -38,7 +39,8 @@
       <div class="content-area-header">
         <h2>Tags</h2>
       </div>
-      <div class="tag-cloud-container">
+      <div ld-fetch="{{ tag_cloud.refresh_url }}" ld-on="refresh-tag-cloud"
+           class="tag-cloud-container">
         {% include 'bookmarks/tag_cloud.html' %}
       </div>
     </section>

bookmarks/templates/bookmarks/read.html

@@ -0,0 +1,83 @@
+{% load sass_tags %}
+{% load static %}
+<!DOCTYPE html>
+<html lang="en" class="reader-mode">
+<head>
+  <meta charset="UTF-8">
+  <title>Reader view</title>
+  <meta name="viewport" content="width=device-width, initial-scale=1.0, minimal-ui">
+  {% if request.user_profile.theme == 'light' %}
+    <link href="{% sass_src 'theme-light.scss' %}?v={{ app_version }}" rel="stylesheet" type="text/css"/>
+  {% elif request.user_profile.theme == 'dark' %}
+    <link href="{% sass_src 'theme-dark.scss' %}?v={{ app_version }}" rel="stylesheet" type="text/css"/>
+  {% else %}
+    {# Use auto theme as fallback #}
+    <link href="{% sass_src 'theme-dark.scss' %}?v={{ app_version }}" rel="stylesheet" type="text/css"
+          media="(prefers-color-scheme: dark)"/>
+    <link href="{% sass_src 'theme-light.scss' %}?v={{ app_version }}" rel="stylesheet" type="text/css"
+          media="(prefers-color-scheme: light)"/>
+  {% endif %}
+</head>
+<body>
+<template id="content">{{ content|safe }}</template>
+<script src="{% static 'vendor/Readability.js' %}" type="application/javascript"></script>
+<script type="application/javascript">
+  function estimateReadingTime(charCount, wordsPerMinute) {
+    const avgWordLength = 5;
+    const totalWords = charCount / avgWordLength;
+    return Math.ceil(totalWords / wordsPerMinute);
+  }
+
+  function postProcess(articleContent) {
+    articleContent.querySelectorAll('table').forEach(table => {
+      table.classList.add('table');
+    });
+  }
+
+  function makeReadable() {
+    const content = document.getElementById('content');
+    const contentHtml = content.innerHTML;
+    const dom = new DOMParser().parseFromString(contentHtml, 'text/html');
+    const article = new Readability(dom).parse();
+
+    document.title = article.title;
+
+    const container = document.createElement('div');
+    container.classList.add('container');
+
+    const articleTitle = document.createElement('h1');
+    articleTitle.textContent = article.title;
+    container.append(articleTitle);
+
+    const byline = [article.byline, article.siteName].filter(Boolean);
+    if (byline.length > 0) {
+      const articleByline = document.createElement('p');
+      articleByline.textContent = byline.join(' | ');
+      articleByline.classList.add('byline');
+      container.append(articleByline);
+    }
+
+    if(article.length) {
+      const minTime = estimateReadingTime(article.length, 225);
+      const maxTime = estimateReadingTime(article.length, 175);
+
+      const articleReadingTime = document.createElement('p');
+      articleReadingTime.textContent = `${minTime}-${maxTime} minutes`;
+      articleReadingTime.classList.add('reading-time');
+      container.append(articleReadingTime);
+    }
+
+    const divider = document.createElement('hr');
+    container.append(divider);
+
+    const articleContent = document.createElement('div');
+    articleContent.innerHTML = article.content;
+    postProcess(articleContent);
+    container.append(articleContent);
+
+    content.replaceWith(container);
+  }
+  makeReadable();
+</script>
+</body>
+</html>

bookmarks/templates/bookmarks/shared.html

@@ -4,10 +4,7 @@
 {% load bookmarks %}
 
 {% block content %}
-  <div class="bookmarks-page grid columns-md-1"
-       ld-bookmark-page
-       bookmarks-url="{% url 'bookmarks:partials.bookmark_list.shared' %}"
-       tags-url="{% url 'bookmarks:partials.tag_cloud.shared' %}">
+  <div class="bookmarks-page grid columns-md-1">
 
     {# Bookmark list #}
     <section class="content-area col-2">
@@ -15,15 +12,20 @@
         <h2>Shared bookmarks</h2>
         <div class="header-controls">
           {% bookmark_search bookmark_list.search tag_cloud.tags mode='shared' %}
-          <button ld-modal modal-content=".tag-cloud" class="btn ml-2 show-md">Tags</button>
+          <button ld-fetch="{{ bookmark_list.tag_modal_url }}" ld-target="body|append" ld-on="click"
+                  class="btn ml-2 show-md">Tags
+          </button>
         </div>
       </div>
 
-      <form class="bookmark-actions" action="{{ bookmark_list.action_url|safe }}"
-            method="post">
+      <form ld-form ld-fire="refresh-bookmark-list,refresh-tag-cloud"
+            class="bookmark-actions"
+            action="{{ bookmark_list.action_url|safe }}"
+            method="post" autocomplete="off">
         {% csrf_token %}
-
-        <div class="bookmark-list-container">
+        <div ld-fetch="{{ bookmark_list.refresh_url }}" ld-on="refresh-bookmark-list"
+             ld-fire="refresh-bookmark-list-done"
+             class="bookmark-list-container">
           {% include 'bookmarks/bookmark_list.html' %}
         </div>
       </form>
@@ -41,7 +43,8 @@
       <div class="content-area-header">
         <h2>Tags</h2>
       </div>
-      <div class="tag-cloud-container">
+      <div ld-fetch="{{ tag_cloud.refresh_url }}" ld-on="refresh-tag-cloud"
+           class="tag-cloud-container">
         {% include 'bookmarks/tag_cloud.html' %}
       </div>
     </section>

bookmarks/templates/bookmarks/tag_modal.html

@@ -0,0 +1,21 @@
+<div ld-modal class="modal active">
+  <div class="modal-overlay" aria-label="Close"></div>
+  <div class="modal-container">
+    <div class="modal-header d-flex justify-between align-center">
+      <div class="modal-title h5">Tags</div>
+      <button class="close">
+        <svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" stroke-width="2"
+             stroke="currentColor" fill="none" stroke-linecap="round" stroke-linejoin="round">
+          <path stroke="none" d="M0 0h24v24H0z" fill="none"></path>
+          <path d="M18 6l-12 12"></path>
+          <path d="M6 6l12 12"></path>
+        </svg>
+      </button>
+    </div>
+    <div class="modal-body">
+      <div class="content">
+        {% include 'bookmarks/tag_cloud.html' %}
+      </div>
+    </div>
+  </div>
+</div>

bookmarks/templates/settings/general.html

@@ -8,6 +8,12 @@
 
     {# Profile section #}
     <section class="content-area">
+      {% if success_message %}
+        <div class="toast toast-success mb-4">{{ success_message }}</div>
+      {% endif %}
+      {% if error_message %}
+        <div class="toast toast-error mb-4">{{ error_message }}</div>
+      {% endif %}
       <h2>Profile</h2>
       <p>
         <a href="{% url 'change_password' %}">Change password</a>
@@ -104,6 +110,29 @@
             result will also include bookmarks where a search term matches otherwise.
           </div>
         </div>
+        <div class="form-group">
+          <label for="{{ form.tag_grouping.id_for_label }}" class="form-label">Tag grouping</label>
+          {{ form.tag_grouping|add_class:"form-select width-25 width-sm-100" }}
+          <div class="form-input-hint">
+            In alphabetical mode, tags will be grouped by the first letter.
+            If disabled, tags will not be grouped.
+          </div>
+        </div>
+        <div class="form-group">
+          <details {% if form.auto_tagging_rules.value %}open{% endif %}>
+            <summary>Auto Tagging</summary>
+            <label for="{{ form.auto_tagging_rules.id_for_label }}" class="text-assistive">Auto Tagging</label>
+            <div class="mt-2">
+              {{ form.auto_tagging_rules|add_class:"form-input custom-css"|attr:"rows:6" }}
+            </div>
+          </details>
+          <div class="form-input-hint">
+            Automatically adds tags to bookmarks based on predefined rules.
+            Each line is a single rule that maps a URL to one or more tags. For example:
+            <pre>youtube.com video
+reddit.com/r/Music music reddit</pre>
+          </div>
+        </div>
         <div class="form-group">
           <label for="{{ form.enable_favicons.id_for_label }}" class="form-checkbox">
             {{ form.enable_favicons }}
@@ -111,6 +140,7 @@
           </label>
           <div class="form-input-hint">
             Automatically loads favicons for bookmarked websites and displays them next to each bookmark.
+            Enabling this feature automatically downloads all missing favicons.
             By default, this feature uses a <b>Google service</b> to download favicons.
             If you don't want to use this service, check the <a
               href="https://github.com/sissbruecker/linkding/blob/master/docs/Options.md#ld_favicon_provider"
@@ -120,13 +150,16 @@
           {% if request.user_profile.enable_favicons and enable_refresh_favicons %}
             <button class="btn mt-2" name="refresh_favicons">Refresh Favicons</button>
           {% endif %}
-          {% if refresh_favicons_success_message %}
-            <div class="has-success">
-              <p class="form-input-hint">
-                {{ refresh_favicons_success_message }}
-              </p>
-            </div>
-          {% endif %}
+        </div>
+        <div class="form-group">
+          <label for="{{ form.enable_preview_images.id_for_label }}" class="form-checkbox">
+            {{ form.enable_preview_images }}
+            <i class="form-icon"></i> Enable Preview Images
+          </label>
+          <div class="form-input-hint">
+            Automatically loads preview images for bookmarked websites and displays them next to each bookmark.
+            Enabling this feature automatically downloads all missing preview images.
+          </div>
         </div>
         <div class="form-group">
           <label for="{{ form.web_archive_integration.id_for_label }}" class="form-label">Internet Archive
@@ -173,8 +206,20 @@
               Automatically creates HTML snapshots when adding bookmarks. Alternatively, when disabled, snapshots can be
               created manually in the details view of a bookmark.
             </div>
+            <button class="btn mt-2" name="create_missing_html_snapshots">Create missing HTML snapshots</button>
           </div>
         {% endif %}
+        <div class="form-group">
+          <label for="{{ form.default_mark_unread.id_for_label }}" class="form-checkbox">
+            {{ form.default_mark_unread }}
+            <i class="form-icon"></i> Create bookmarks as unread by default
+          </label>
+          <div class="form-input-hint">
+            Sets the default state for the "Mark as unread" option when creating a new bookmark.
+            Setting this option will make all new bookmarks default to unread.
+            This can be overridden when creating each new bookmark.
+          </div>
+        </div>
         <div class="form-group">
           <details {% if form.custom_css.value %}open{% endif %}>
             <summary>Custom CSS</summary>
@@ -189,13 +234,6 @@
         </div>
         <div class="form-group">
           <input type="submit" name="update_profile" value="Save" class="btn btn-primary mt-2">
-          {% if update_profile_success_message %}
-            <div class="has-success">
-              <p class="form-input-hint">
-                {{ update_profile_success_message }}
-              </p>
-            </div>
-          {% endif %}
         </div>
       </form>
     </section>
@@ -224,20 +262,6 @@
             <input class="form-input" type="file" name="import_file">
             <input type="submit" class="input-group-btn btn btn-primary" value="Upload">
           </div>
-          {% if import_success_message %}
-            <div class="has-success">
-              <p class="form-input-hint">
-                {{ import_success_message }}
-              </p>
-            </div>
-          {% endif %}
-          {% if import_errors_message %}
-            <div class="has-error">
-              <p class="form-input-hint">
-                {{ import_errors_message }}
-              </p>
-            </div>
-          {% endif %}
         </div>
       </form>
     </section>

bookmarks/tests/helpers.py

@@ -39,6 +39,7 @@ class BookmarkFactoryMixin:
         website_description: str = "",
         web_archive_snapshot_url: str = "",
         favicon_file: str = "",
+        preview_image_file: str = "",
         added: datetime = None,
     ):
         if title is None:
@@ -67,6 +68,7 @@ class BookmarkFactoryMixin:
             shared=shared,
             web_archive_snapshot_url=web_archive_snapshot_url,
             favicon_file=favicon_file,
+            preview_image_file=preview_image_file,
         )
         bookmark.save()
         for tag in tags:

bookmarks/tests/test_auto_tagging.py

@@ -0,0 +1,179 @@
+from bookmarks.services import auto_tagging
+from django.test import TestCase
+
+
+class AutoTaggingTestCase(TestCase):
+    def test_auto_tag_by_domain(self):
+        script = """
+            example.com example
+            test.com test
+        """
+        url = "https://example.com/"
+
+        tags = auto_tagging.get_tags(script, url)
+
+        self.assertEqual(tags, set(["example"]))
+
+    def test_auto_tag_by_domain_ignores_case(self):
+        script = """
+            EXAMPLE.com example
+        """
+        url = "https://example.com/"
+
+        tags = auto_tagging.get_tags(script, url)
+
+        self.assertEqual(tags, set(["example"]))
+
+    def test_auto_tag_by_domain_should_add_all_tags(self):
+        script = """
+            example.com one two three
+        """
+        url = "https://example.com/"
+
+        tags = auto_tagging.get_tags(script, url)
+
+        self.assertEqual(tags, set(["one", "two", "three"]))
+
+    def test_auto_tag_by_domain_work_with_idn_domains(self):
+        script = """
+            रजिस्ट्री.भारत tag1
+        """
+        url = "https://www.xn--81bg3cc2b2bk5hb.xn--h2brj9c/"
+
+        tags = auto_tagging.get_tags(script, url)
+
+        self.assertEqual(tags, set(["tag1"]))
+
+        script = """
+            xn--81bg3cc2b2bk5hb.xn--h2brj9c tag1
+        """
+        url = "https://www.रजिस्ट्री.भारत/"
+
+        tags = auto_tagging.get_tags(script, url)
+
+        self.assertEqual(tags, set(["tag1"]))
+
+    def test_auto_tag_by_domain_and_path(self):
+        script = """
+            example.com/one one
+            example.com/two two
+            test.com test
+        """
+        url = "https://example.com/one/"
+
+        tags = auto_tagging.get_tags(script, url)
+
+        self.assertEqual(tags, set(["one"]))
+
+    def test_auto_tag_by_domain_and_path_ignores_case(self):
+        script = """
+            example.com/One one
+        """
+        url = "https://example.com/one/"
+
+        tags = auto_tagging.get_tags(script, url)
+
+        self.assertEqual(tags, set(["one"]))
+
+    def test_auto_tag_by_domain_and_path_matches_path_ltr(self):
+        script = """
+            example.com/one one
+            example.com/two two
+            test.com test
+        """
+        url = "https://example.com/one/two"
+
+        tags = auto_tagging.get_tags(script, url)
+
+        self.assertEqual(tags, set(["one"]))
+
+    def test_auto_tag_by_domain_ignores_domain_in_path(self):
+        script = """
+            example.com example
+        """
+        url = "https://test.com/example.com"
+
+        tags = auto_tagging.get_tags(script, url)
+
+        self.assertEqual(tags, set([]))
+
+    def test_auto_tag_by_domain_includes_subdomains(self):
+        script = """
+            example.com example
+            test.example.com test
+            some.example.com some
+        """
+        url = "https://test.example.com/"
+
+        tags = auto_tagging.get_tags(script, url)
+
+        self.assertEqual(tags, set(["example", "test"]))
+
+    def test_auto_tag_by_domain_matches_domain_rtl(self):
+        script = """
+            example.com example
+        """
+        url = "https://example.com.bad-website.com/"
+
+        tags = auto_tagging.get_tags(script, url)
+
+        self.assertEqual(tags, set([]))
+
+    def test_auto_tag_by_domain_ignores_schema(self):
+        script = """
+            https://example.com/ https
+            http://example.com/ http
+        """
+        url = "http://example.com/"
+
+        tags = auto_tagging.get_tags(script, url)
+
+        self.assertEqual(tags, set(["https", "http"]))
+
+    def test_auto_tag_by_domain_ignores_lines_with_no_tags(self):
+        script = """
+            example.com
+        """
+        url = "https://example.com/"
+
+        tags = auto_tagging.get_tags(script, url)
+
+        self.assertEqual(tags, set([]))
+
+    def test_auto_tag_by_domain_path_and_qs(self):
+        script = """
+            example.com/page?a=b tag1     # true, matches a=b
+            example.com/page?a=c&c=d tag2 # true, matches both a=c and c=d
+            example.com/page?c=d&l=p tag3 # false, l=p doesn't exists
+            example.com/page?a=bb tag4    # false bb != b
+            example.com/page?a=b&a=c tag5 # true, matches both a=b and a=c
+            example.com/page?a=B tag6     # true, matches a=b because case insensitive
+            example.com/page?A=b tag7     # true, matches a=b because case insensitive
+        """
+        url = "https://example.com/page/some?z=x&a=b&v=b&c=d&o=p&a=c"
+
+        tags = auto_tagging.get_tags(script, url)
+
+        self.assertEqual(tags, set(["tag1", "tag2", "tag5", "tag6", "tag7"]))
+
+    def test_auto_tag_by_domain_path_and_qs_with_empty_value(self):
+        script = """
+            example.com/page?a= tag1
+            example.com/page?b= tag2
+        """
+        url = "https://example.com/page/some?a=value"
+
+        tags = auto_tagging.get_tags(script, url)
+
+        self.assertEqual(tags, set(["tag1"]))
+
+    def test_auto_tag_by_domain_path_and_qs_works_with_encoded_url(self):
+        script = """
+            example.com/page?a=йцу tag1
+            example.com/page?a=%D0%B9%D1%86%D1%83 tag2
+        """
+        url = "https://example.com/page?a=%D0%B9%D1%86%D1%83"
+
+        tags = auto_tagging.get_tags(script, url)
+
+        self.assertEqual(tags, set(["tag1", "tag2"]))

bookmarks/tests/test_bookmark_archived_view.py

@@ -94,15 +94,10 @@ class BookmarkArchivedViewTestCase(TestCase, BookmarkFactoryMixin, HtmlTestMixin
         )
 
     def assertBulkActionForm(self, response, url: str):
-        html = collapse_whitespace(response.content.decode())
-        needle = collapse_whitespace(
-            f"""
-            <form class="bookmark-actions"
-                action="{url}"
-                method="post" autocomplete="off">
-        """
-        )
-        self.assertIn(needle, html)
+        soup = self.make_soup(response.content.decode())
+        form = soup.select_one("form.bookmark-actions")
+        self.assertIsNotNone(form)
+        self.assertEqual(form.attrs["action"], url)
 
     def test_should_list_archived_and_user_owned_bookmarks(self):
         other_user = User.objects.create_user(

bookmarks/tests/test_bookmark_asset_view.py

@@ -34,12 +34,12 @@ class BookmarkAssetViewTestCase(TestCase, BookmarkFactoryMixin):
         asset = self.setup_asset(bookmark=bookmark, file=filename)
         return asset
 
-    def test_view_access(self):
+    def view_access_test(self, view_name: str):
         # own bookmark
         bookmark = self.setup_bookmark()
         asset = self.setup_asset_with_file(bookmark)
 
-        response = self.client.get(reverse("bookmarks:assets.view", args=[asset.id]))
+        response = self.client.get(reverse(view_name, args=[asset.id]))
         self.assertEqual(response.status_code, 200)
 
         # other user's bookmark
@@ -47,14 +47,14 @@ class BookmarkAssetViewTestCase(TestCase, BookmarkFactoryMixin):
         bookmark = self.setup_bookmark(user=other_user)
         asset = self.setup_asset_with_file(bookmark)
 
-        response = self.client.get(reverse("bookmarks:assets.view", args=[asset.id]))
+        response = self.client.get(reverse(view_name, args=[asset.id]))
         self.assertEqual(response.status_code, 404)
 
         # shared, sharing disabled
         bookmark = self.setup_bookmark(user=other_user, shared=True)
         asset = self.setup_asset_with_file(bookmark)
 
-        response = self.client.get(reverse("bookmarks:assets.view", args=[asset.id]))
+        response = self.client.get(reverse(view_name, args=[asset.id]))
         self.assertEqual(response.status_code, 404)
 
         # unshared, sharing enabled
@@ -64,31 +64,31 @@ class BookmarkAssetViewTestCase(TestCase, BookmarkFactoryMixin):
         bookmark = self.setup_bookmark(user=other_user, shared=False)
         asset = self.setup_asset_with_file(bookmark)
 
-        response = self.client.get(reverse("bookmarks:assets.view", args=[asset.id]))
+        response = self.client.get(reverse(view_name, args=[asset.id]))
         self.assertEqual(response.status_code, 404)
 
         # shared, sharing enabled
         bookmark = self.setup_bookmark(user=other_user, shared=True)
         asset = self.setup_asset_with_file(bookmark)
 
-        response = self.client.get(reverse("bookmarks:assets.view", args=[asset.id]))
+        response = self.client.get(reverse(view_name, args=[asset.id]))
         self.assertEqual(response.status_code, 200)
 
-    def test_view_access_guest_user(self):
+    def view_access_guest_user_test(self, view_name: str):
         self.client.logout()
 
         # unshared, sharing disabled
         bookmark = self.setup_bookmark()
         asset = self.setup_asset_with_file(bookmark)
 
-        response = self.client.get(reverse("bookmarks:assets.view", args=[asset.id]))
+        response = self.client.get(reverse(view_name, args=[asset.id]))
         self.assertEqual(response.status_code, 404)
 
         # shared, sharing disabled
         bookmark = self.setup_bookmark(shared=True)
         asset = self.setup_asset_with_file(bookmark)
 
-        response = self.client.get(reverse("bookmarks:assets.view", args=[asset.id]))
+        response = self.client.get(reverse(view_name, args=[asset.id]))
         self.assertEqual(response.status_code, 404)
 
         # unshared, sharing enabled
@@ -98,14 +98,14 @@ class BookmarkAssetViewTestCase(TestCase, BookmarkFactoryMixin):
         bookmark = self.setup_bookmark(shared=False)
         asset = self.setup_asset_with_file(bookmark)
 
-        response = self.client.get(reverse("bookmarks:assets.view", args=[asset.id]))
+        response = self.client.get(reverse(view_name, args=[asset.id]))
         self.assertEqual(response.status_code, 404)
 
         # shared, sharing enabled
         bookmark = self.setup_bookmark(shared=True)
         asset = self.setup_asset_with_file(bookmark)
 
-        response = self.client.get(reverse("bookmarks:assets.view", args=[asset.id]))
+        response = self.client.get(reverse(view_name, args=[asset.id]))
         self.assertEqual(response.status_code, 404)
 
         # unshared, public sharing enabled
@@ -114,12 +114,24 @@ class BookmarkAssetViewTestCase(TestCase, BookmarkFactoryMixin):
         bookmark = self.setup_bookmark(shared=False)
         asset = self.setup_asset_with_file(bookmark)
 
-        response = self.client.get(reverse("bookmarks:assets.view", args=[asset.id]))
+        response = self.client.get(reverse(view_name, args=[asset.id]))
         self.assertEqual(response.status_code, 404)
 
         # shared, public sharing enabled
         bookmark = self.setup_bookmark(shared=True)
         asset = self.setup_asset_with_file(bookmark)
 
-        response = self.client.get(reverse("bookmarks:assets.view", args=[asset.id]))
+        response = self.client.get(reverse(view_name, args=[asset.id]))
         self.assertEqual(response.status_code, 200)
+
+    def test_view_access(self):
+        self.view_access_test("bookmarks:assets.view")
+
+    def test_view_access_guest_user(self):
+        self.view_access_guest_user_test("bookmarks:assets.view")
+
+    def test_reader_view_access(self):
+        self.view_access_test("bookmarks:assets.read")
+
+    def test_reader_view_access_guest_user(self):
+        self.view_access_guest_user_test("bookmarks:assets.read")

bookmarks/tests/test_bookmark_details_modal.py

@@ -1,11 +1,13 @@
+import re
 from unittest.mock import patch
 
+from django.core.files.uploadedfile import SimpleUploadedFile
 from django.test import TestCase, override_settings
 from django.urls import reverse
 from django.utils import formats
 
 from bookmarks.models import BookmarkAsset, UserProfile
-from bookmarks.services import tasks
+from bookmarks.services import bookmarks, tasks
 from bookmarks.tests.helpers import BookmarkFactoryMixin, HtmlTestMixin
 
 
@@ -45,6 +47,9 @@ class BookmarkDetailsModalTestCase(TestCase, BookmarkFactoryMixin, HtmlTestMixin
     def find_weblink(self, soup, url):
         return soup.find("a", {"class": "weblink", "href": url})
 
+    def count_weblinks(self, soup):
+        return len(soup.find_all("a", {"class": "weblink"}))
+
     def find_asset(self, soup, asset):
         return soup.find("div", {"data-asset-id": asset.id})
 
@@ -105,17 +110,11 @@ class BookmarkDetailsModalTestCase(TestCase, BookmarkFactoryMixin, HtmlTestMixin
     def test_access_with_sharing(self):
         self.details_route_sharing_access_test(self.get_view_name(), True)
 
-    def test_form_partial_access(self):
-        # form partial is only used when submitting forms, which should be only
-        # accessible to the owner of the bookmark. As such assume it requires
-        # login.
-        self.details_route_access_test("bookmarks:partials.details_form", False)
+    def test_assets_access(self):
+        self.details_route_access_test("bookmarks:details_assets", True)
 
-    def test_form_partial_access_with_sharing(self):
-        # form partial is only used when submitting forms, which should be only
-        # accessible to the owner of the bookmark. As such assume it requires
-        # login.
-        self.details_route_sharing_access_test("bookmarks:partials.details_form", False)
+    def test_assets_access_with_sharing(self):
+        self.details_route_sharing_access_test("bookmarks:details_assets", True)
 
     def test_displays_title(self):
         # with title
@@ -177,6 +176,48 @@ class BookmarkDetailsModalTestCase(TestCase, BookmarkFactoryMixin, HtmlTestMixin
         self.assertIsNotNone(image)
         self.assertEqual(image["src"], "/static/example.png")
 
+    def test_reader_mode_link(self):
+        # no latest snapshot
+        bookmark = self.setup_bookmark()
+        soup = self.get_details(bookmark)
+        self.assertEqual(self.count_weblinks(soup), 1)
+
+        # snapshot is not complete
+        self.setup_asset(
+            bookmark,
+            asset_type=BookmarkAsset.TYPE_SNAPSHOT,
+            status=BookmarkAsset.STATUS_PENDING,
+        )
+        self.setup_asset(
+            bookmark,
+            asset_type=BookmarkAsset.TYPE_SNAPSHOT,
+            status=BookmarkAsset.STATUS_FAILURE,
+        )
+        soup = self.get_details(bookmark)
+        self.assertEqual(self.count_weblinks(soup), 1)
+
+        # not a snapshot
+        self.setup_asset(
+            bookmark,
+            asset_type="upload",
+            status=BookmarkAsset.STATUS_COMPLETE,
+        )
+        soup = self.get_details(bookmark)
+        self.assertEqual(self.count_weblinks(soup), 1)
+
+        # snapshot is complete
+        asset = self.setup_asset(
+            bookmark,
+            asset_type=BookmarkAsset.TYPE_SNAPSHOT,
+            status=BookmarkAsset.STATUS_COMPLETE,
+        )
+        soup = self.get_details(bookmark)
+        self.assertEqual(self.count_weblinks(soup), 2)
+
+        reader_mode_url = reverse("bookmarks:assets.read", args=[asset.id])
+        link = self.find_weblink(soup, reader_mode_url)
+        self.assertIsNotNone(link)
+
     def test_internet_archive_link(self):
         # without snapshot url
         bookmark = self.setup_bookmark()
@@ -190,7 +231,7 @@ class BookmarkDetailsModalTestCase(TestCase, BookmarkFactoryMixin, HtmlTestMixin
         link = self.find_weblink(soup, bookmark.web_archive_snapshot_url)
         self.assertIsNotNone(link)
         self.assertEqual(link["href"], bookmark.web_archive_snapshot_url)
-        self.assertEqual(link.text.strip(), "View on Internet Archive")
+        self.assertEqual(link.text.strip(), "Internet Archive")
 
         # favicons disabled
         bookmark = self.setup_bookmark(
@@ -259,6 +300,36 @@ class BookmarkDetailsModalTestCase(TestCase, BookmarkFactoryMixin, HtmlTestMixin
             web_archive_link["target"], UserProfile.BOOKMARK_LINK_TARGET_SELF
         )
 
+    def test_preview_image(self):
+        # without image
+        bookmark = self.setup_bookmark()
+        soup = self.get_details(bookmark)
+        image = soup.select_one("div.preview-image img")
+        self.assertIsNone(image)
+
+        # with image
+        bookmark = self.setup_bookmark(preview_image_file="example.png")
+        soup = self.get_details(bookmark)
+        image = soup.select_one("div.preview-image img")
+        self.assertIsNone(image)
+
+        # preview images enabled, no image
+        profile = self.get_or_create_test_user().profile
+        profile.enable_preview_images = True
+        profile.save()
+
+        bookmark = self.setup_bookmark()
+        soup = self.get_details(bookmark)
+        image = soup.select_one("div.preview-image img")
+        self.assertIsNone(image)
+
+        # preview images enabled, image present
+        bookmark = self.setup_bookmark(preview_image_file="example.png")
+        soup = self.get_details(bookmark)
+        image = soup.select_one("div.preview-image img")
+        self.assertIsNotNone(image)
+        self.assertEqual(image["src"], "/static/example.png")
+
     def test_status(self):
         # renders form
         bookmark = self.setup_bookmark()
@@ -765,6 +836,27 @@ class BookmarkDetailsModalTestCase(TestCase, BookmarkFactoryMixin, HtmlTestMixin
         self.assertEqual(response.status_code, 404)
         self.assertTrue(BookmarkAsset.objects.filter(id=asset.id).exists())
 
+    @override_settings(LD_ENABLE_SNAPSHOTS=True)
+    def test_assets_refresh_when_having_pending_asset(self):
+        bookmark = self.setup_bookmark()
+        asset = self.setup_asset(bookmark, status=BookmarkAsset.STATUS_COMPLETE)
+        fetch_url = reverse("bookmarks:details_assets", args=[bookmark.id])
+
+        # no pending asset
+        soup = self.get_details(bookmark)
+        files_section = self.find_section(soup, "Files")
+        assets_wrapper = files_section.find("div", {"ld-fetch": fetch_url})
+        self.assertIsNone(assets_wrapper)
+
+        # with pending asset
+        asset.status = BookmarkAsset.STATUS_PENDING
+        asset.save()
+
+        soup = self.get_details(bookmark)
+        files_section = self.find_section(soup, "Files")
+        assets_wrapper = files_section.find("div", {"ld-fetch": fetch_url})
+        self.assertIsNotNone(assets_wrapper)
+
     @override_settings(LD_ENABLE_SNAPSHOTS=True)
     def test_create_snapshot(self):
         with patch.object(
@@ -776,6 +868,59 @@ class BookmarkDetailsModalTestCase(TestCase, BookmarkFactoryMixin, HtmlTestMixin
             )
             self.assertEqual(response.status_code, 302)
 
-            mock_create_html_snapshot_task.assert_called_with(bookmark.id)
-
             self.assertEqual(bookmark.bookmarkasset_set.count(), 1)
+
+    @override_settings(LD_ENABLE_SNAPSHOTS=True)
+    def test_create_snapshot_is_disabled_when_having_pending_asset(self):
+        bookmark = self.setup_bookmark()
+        asset = self.setup_asset(bookmark, status=BookmarkAsset.STATUS_COMPLETE)
+
+        # no pending asset
+        soup = self.get_details(bookmark)
+        files_section = self.find_section(soup, "Files")
+        create_button = files_section.find(
+            "button", string=re.compile("Create HTML snapshot")
+        )
+        self.assertFalse(create_button.has_attr("disabled"))
+
+        # with pending asset
+        asset.status = BookmarkAsset.STATUS_PENDING
+        asset.save()
+
+        soup = self.get_details(bookmark)
+        files_section = self.find_section(soup, "Files")
+        create_button = files_section.find(
+            "button", string=re.compile("Create HTML snapshot")
+        )
+        self.assertTrue(create_button.has_attr("disabled"))
+
+    def test_upload_file(self):
+        bookmark = self.setup_bookmark()
+        file_content = b"file content"
+        upload_file = SimpleUploadedFile("test.txt", file_content)
+
+        with patch.object(bookmarks, "upload_asset") as mock_upload_asset:
+            response = self.client.post(
+                self.get_base_url(bookmark),
+                {"upload_asset": "", "upload_asset_file": upload_file},
+            )
+            self.assertEqual(response.status_code, 302)
+
+            mock_upload_asset.assert_called_once()
+
+            args, kwargs = mock_upload_asset.call_args
+            self.assertEqual(args[0], bookmark)
+
+            upload_file = args[1]
+            self.assertEqual(upload_file.name, "test.txt")
+
+    def test_upload_file_without_file(self):
+        bookmark = self.setup_bookmark()
+
+        with patch.object(bookmarks, "upload_asset") as mock_upload_asset:
+            response = self.client.post(
+                self.get_base_url(bookmark),
+                {"upload_asset": ""},
+            )
+            self.assertEqual(response.status_code, 400)
+            mock_upload_asset.assert_not_called()

bookmarks/tests/test_bookmark_index_view.py

@@ -94,15 +94,10 @@ class BookmarkIndexViewTestCase(TestCase, BookmarkFactoryMixin, HtmlTestMixin):
         )
 
     def assertBulkActionForm(self, response, url: str):
-        html = collapse_whitespace(response.content.decode())
-        needle = collapse_whitespace(
-            f"""
-            <form class="bookmark-actions"
-                action="{url}"
-                method="post" autocomplete="off">
-        """
-        )
-        self.assertIn(needle, html)
+        soup = self.make_soup(response.content.decode())
+        form = soup.select_one("form.bookmark-actions")
+        self.assertIsNotNone(form)
+        self.assertEqual(form.attrs["action"], url)
 
     def test_should_list_unarchived_and_user_owned_bookmarks(self):
         other_user = User.objects.create_user(

bookmarks/tests/test_bookmark_new_view.py

@@ -210,3 +210,25 @@ class BookmarkNewViewTestCase(TestCase, BookmarkFactoryMixin):
         response = self.client.get(reverse("bookmarks:edit", args=[bookmark.id]))
 
         self.assertContains(response, '<details class="notes">', count=1)
+
+    def test_should_not_check_unread_by_default(self):
+        response = self.client.get(reverse("bookmarks:new"))
+        html = response.content.decode()
+
+        self.assertInHTML(
+            '<input type="checkbox" name="unread" id="id_unread">',
+            html,
+        )
+
+    def test_should_check_unread_when_configured_in_profile(self):
+        self.user.profile.default_mark_unread = True
+        self.user.profile.save()
+
+        response = self.client.get(reverse("bookmarks:new"))
+        html = response.content.decode()
+
+        self.assertInHTML(
+            '<input type="checkbox" name="unread" value="true" '
+            'id="id_unread" checked="">',
+            html,
+        )

bookmarks/tests/test_bookmarks_api.py

@@ -440,6 +440,20 @@ class BookmarksApiTestCase(LinkdingApiTestCase, BookmarkFactoryMixin):
         bookmark = Bookmark.objects.get(url=data["url"])
         self.assertFalse(bookmark.shared)
 
+    def test_create_bookmark_should_add_tags_from_auto_tagging(self):
+        tag1 = self.setup_tag()
+        tag2 = self.setup_tag()
+
+        self.authenticate()
+        profile = self.get_or_create_test_user().profile
+        profile.auto_tagging_rules = f"example.com {tag2.name}"
+        profile.save()
+
+        data = {"url": "https://example.com/", "tag_names": [tag1.name]}
+        self.post(reverse("bookmarks:bookmark-list"), data, status.HTTP_201_CREATED)
+        bookmark = Bookmark.objects.get(url=data["url"])
+        self.assertCountEqual(bookmark.tags.all(), [tag1, tag2])
+
     def test_get_bookmark(self):
         self.authenticate()
         bookmark = self.setup_bookmark()
@@ -512,6 +526,22 @@ class BookmarksApiTestCase(LinkdingApiTestCase, BookmarkFactoryMixin):
         updated_bookmark = Bookmark.objects.get(id=bookmark.id)
         self.assertEqual(updated_bookmark.shared, True)
 
+    def test_update_bookmark_adds_tags_from_auto_tagging(self):
+        bookmark = self.setup_bookmark()
+        tag1 = self.setup_tag()
+        tag2 = self.setup_tag()
+
+        self.authenticate()
+        profile = self.get_or_create_test_user().profile
+        profile.auto_tagging_rules = f"example.com {tag2.name}"
+        profile.save()
+
+        data = {"url": "https://example.com/", "tag_names": [tag1.name]}
+        url = reverse("bookmarks:bookmark-detail", args=[bookmark.id])
+        self.put(url, data, expected_status_code=status.HTTP_200_OK)
+        updated_bookmark = Bookmark.objects.get(id=bookmark.id)
+        self.assertCountEqual(updated_bookmark.tags.all(), [tag1, tag2])
+
     def test_patch_bookmark(self):
         self.authenticate()
         bookmark = self.setup_bookmark()
@@ -583,6 +613,22 @@ class BookmarksApiTestCase(LinkdingApiTestCase, BookmarkFactoryMixin):
         self.assertEqual(updated_bookmark.description, bookmark.description)
         self.assertListEqual(updated_bookmark.tag_names, bookmark.tag_names)
 
+    def test_patch_bookmark_adds_tags_from_auto_tagging(self):
+        bookmark = self.setup_bookmark()
+        tag1 = self.setup_tag()
+        tag2 = self.setup_tag()
+
+        self.authenticate()
+        profile = self.get_or_create_test_user().profile
+        profile.auto_tagging_rules = f"example.com {tag2.name}"
+        profile.save()
+
+        data = {"tag_names": [tag1.name]}
+        url = reverse("bookmarks:bookmark-detail", args=[bookmark.id])
+        self.patch(url, data, expected_status_code=status.HTTP_200_OK)
+        updated_bookmark = Bookmark.objects.get(id=bookmark.id)
+        self.assertCountEqual(updated_bookmark.tags.all(), [tag1, tag2])
+
     def test_delete_bookmark(self):
         self.authenticate()
         bookmark = self.setup_bookmark()
@@ -628,7 +674,10 @@ class BookmarksApiTestCase(LinkdingApiTestCase, BookmarkFactoryMixin):
             website_loader, "load_website_metadata"
         ) as mock_load_website_metadata:
             expected_metadata = WebsiteMetadata(
-                "https://example.com", "Scraped metadata", "Scraped description"
+                "https://example.com",
+                "Scraped metadata",
+                "Scraped description",
+                "https://example.com/preview.png",
             )
             mock_load_website_metadata.return_value = expected_metadata
 
@@ -640,9 +689,10 @@ class BookmarksApiTestCase(LinkdingApiTestCase, BookmarkFactoryMixin):
             metadata = response.data["metadata"]
 
             self.assertIsNotNone(metadata)
-            self.assertIsNotNone(expected_metadata.url, metadata["url"])
-            self.assertIsNotNone(expected_metadata.title, metadata["title"])
-            self.assertIsNotNone(expected_metadata.description, metadata["description"])
+            self.assertEqual(expected_metadata.url, metadata["url"])
+            self.assertEqual(expected_metadata.title, metadata["title"])
+            self.assertEqual(expected_metadata.description, metadata["description"])
+            self.assertEqual(expected_metadata.preview_image, metadata["preview_image"])
 
     def test_check_returns_bookmark_if_url_is_bookmarked(self):
         self.authenticate()
@@ -687,9 +737,38 @@ class BookmarksApiTestCase(LinkdingApiTestCase, BookmarkFactoryMixin):
 
             mock_load_website_metadata.assert_not_called()
             self.assertIsNotNone(metadata)
-            self.assertIsNotNone(bookmark.url, metadata["url"])
-            self.assertIsNotNone(bookmark.website_title, metadata["title"])
-            self.assertIsNotNone(bookmark.website_description, metadata["description"])
+            self.assertEqual(bookmark.url, metadata["url"])
+            self.assertEqual(bookmark.website_title, metadata["title"])
+            self.assertEqual(bookmark.website_description, metadata["description"])
+            self.assertIsNone(metadata["preview_image"])
+
+    def test_check_returns_no_auto_tags_if_none_configured(self):
+        self.authenticate()
+
+        url = reverse("bookmarks:bookmark-check")
+        check_url = urllib.parse.quote_plus("https://example.com")
+        response = self.get(
+            f"{url}?url={check_url}", expected_status_code=status.HTTP_200_OK
+        )
+        auto_tags = response.data["auto_tags"]
+
+        self.assertCountEqual(auto_tags, [])
+
+    def test_check_returns_matching_auto_tags(self):
+        self.authenticate()
+
+        profile = self.get_or_create_test_user().profile
+        profile.auto_tagging_rules = "example.com tag1 tag2"
+        profile.save()
+
+        url = reverse("bookmarks:bookmark-check")
+        check_url = urllib.parse.quote_plus("https://example.com")
+        response = self.get(
+            f"{url}?url={check_url}", expected_status_code=status.HTTP_200_OK
+        )
+        auto_tags = response.data["auto_tags"]
+
+        self.assertCountEqual(auto_tags, ["tag1", "tag2"])
 
     def test_can_only_access_own_bookmarks(self):
         self.authenticate()

bookmarks/tests/test_bookmarks_list_template.py

@@ -20,7 +20,7 @@ class BookmarkListTemplateTest(TestCase, BookmarkFactoryMixin, HtmlTestMixin):
         self, html: str, bookmark: Bookmark, link_target: str = "_blank"
     ):
         favicon_img = (
-            f'<img src="/static/{bookmark.favicon_file}" alt="">'
+            f'<img class="favicon" src="/static/{bookmark.favicon_file}" alt="">'
             if bookmark.favicon_file
             else ""
         )
@@ -80,7 +80,9 @@ class BookmarkListTemplateTest(TestCase, BookmarkFactoryMixin, HtmlTestMixin):
         details_modal_url = reverse("bookmarks:details_modal", args=[bookmark.id])
         self.assertInHTML(
             f"""
-                <a ld-modal modal-url="{details_modal_url}?return_url={return_url}" href="{details_url}">View</a>
+                <a ld-fetch="{details_modal_url}?return_url={return_url}" 
+                   ld-on="click" ld-target="body|append" 
+                   href="{details_url}">View</a>
             """,
             html,
             count=count,
@@ -146,19 +148,41 @@ class BookmarkListTemplateTest(TestCase, BookmarkFactoryMixin, HtmlTestMixin):
         )
 
     def assertFaviconVisible(self, html: str, bookmark: Bookmark):
-        self.assertFaviconCount(html, bookmark, 1)
+        self.assertFavicon(html, bookmark, True)
 
     def assertFaviconHidden(self, html: str, bookmark: Bookmark):
-        self.assertFaviconCount(html, bookmark, 0)
+        self.assertFavicon(html, bookmark, False)
 
-    def assertFaviconCount(self, html: str, bookmark: Bookmark, count=1):
-        self.assertInHTML(
-            f"""
-            <img src="/static/{bookmark.favicon_file}" alt="">
-            """,
-            html,
-            count=count,
-        )
+    def assertFavicon(self, html: str, bookmark: Bookmark, visible=True):
+        soup = self.make_soup(html)
+
+        favicon = soup.select_one(".favicon")
+
+        if not visible:
+            self.assertIsNone(favicon)
+            return
+
+        url = f"/static/{bookmark.favicon_file}"
+        self.assertIsNotNone(favicon)
+        self.assertEqual(favicon["src"], url)
+
+    def assertPreviewImageVisible(self, html: str, bookmark: Bookmark):
+        self.assertPreviewImage(html, bookmark, True)
+
+    def assertPreviewImageHidden(self, html: str, bookmark: Bookmark):
+        self.assertPreviewImage(html, bookmark, False)
+
+    def assertPreviewImage(self, html: str, bookmark: Bookmark, visible=True):
+        soup = self.make_soup(html)
+        preview_image = soup.select_one(".preview-image")
+
+        if not visible:
+            self.assertIsNone(preview_image)
+            return
+
+        url = f"/static/{bookmark.preview_image_file}"
+        self.assertIsNotNone(preview_image)
+        self.assertEqual(preview_image["src"], url)
 
     def assertBookmarkURLCount(
         self, html: str, bookmark: Bookmark, link_target: str = "_blank", count=0
@@ -216,7 +240,7 @@ class BookmarkListTemplateTest(TestCase, BookmarkFactoryMixin, HtmlTestMixin):
             f"""
         <button type="submit" name="unshare" value="{bookmark.id}"
                 class="btn btn-link btn-sm btn-icon"
-                ld-confirm-button confirm-icon="ld-icon-unshare" confirm-question="Unshare?">
+                ld-confirm-button ld-confirm-icon="ld-icon-unshare" ld-confirm-question="Unshare?">
           <svg xmlns="http://www.w3.org/2000/svg" width="16" height="16">
             <use xlink:href="#ld-icon-share"></use>
           </svg>
@@ -232,7 +256,7 @@ class BookmarkListTemplateTest(TestCase, BookmarkFactoryMixin, HtmlTestMixin):
             f"""
         <button type="submit" name="mark_as_read" value="{bookmark.id}"
                 class="btn btn-link btn-sm btn-icon"
-                ld-confirm-button confirm-icon="ld-icon-read" confirm-question="Mark as read?">
+                ld-confirm-button ld-confirm-icon="ld-icon-read" ld-confirm-question="Mark as read?">
           <svg xmlns="http://www.w3.org/2000/svg" width="16" height="16">
             <use xlink:href="#ld-icon-unread"></use>
           </svg>
@@ -301,7 +325,7 @@ class BookmarkListTemplateTest(TestCase, BookmarkFactoryMixin, HtmlTestMixin):
         if has_description and has_tags:
             self.assertTrue("|" in description.text)
 
-        # contains description text
+        # contains description text, without leading/trailing whitespace
         if has_description:
             description_text = description.find("span", text=bookmark.description)
             self.assertIsNotNone(description_text)
@@ -370,10 +394,10 @@ class BookmarkListTemplateTest(TestCase, BookmarkFactoryMixin, HtmlTestMixin):
             description = soup.select_one(".description")
             self.assertIsNone(description)
         else:
-            # contains description text
+            # contains description text, without leading/trailing whitespace
             description = soup.select_one(".description.separate")
             self.assertIsNotNone(description)
-            self.assertEqual(description.text.strip(), bookmark.description)
+            self.assertEqual(description.text, bookmark.description)
 
         if not has_tags:
             # no tags element
@@ -638,6 +662,36 @@ class BookmarkListTemplateTest(TestCase, BookmarkFactoryMixin, HtmlTestMixin):
             html,
         )
 
+    def test_preview_image_should_be_visible_when_preview_images_enabled(self):
+        profile = self.get_or_create_test_user().profile
+        profile.enable_preview_images = True
+        profile.save()
+
+        bookmark = self.setup_bookmark(preview_image_file="preview.png")
+        html = self.render_template()
+
+        self.assertPreviewImageVisible(html, bookmark)
+
+    def test_preview_image_should_be_hidden_when_preview_images_disabled(self):
+        profile = self.get_or_create_test_user().profile
+        profile.enable_preview_images = False
+        profile.save()
+
+        bookmark = self.setup_bookmark(preview_image_file="preview.png")
+        html = self.render_template()
+
+        self.assertPreviewImageHidden(html, bookmark)
+
+    def test_preview_image_should_be_hidden_when_there_is_no_preview_image(self):
+        profile = self.get_or_create_test_user().profile
+        profile.enable_preview_images = True
+        profile.save()
+
+        bookmark = self.setup_bookmark()
+        html = self.render_template()
+
+        self.assertPreviewImageHidden(html, bookmark)
+
     def test_favicon_should_be_visible_when_favicons_enabled(self):
         profile = self.get_or_create_test_user().profile
         profile.enable_favicons = True
@@ -782,10 +836,16 @@ class BookmarkListTemplateTest(TestCase, BookmarkFactoryMixin, HtmlTestMixin):
 
     def test_note_cleans_html(self):
         self.setup_bookmark(notes='<script>alert("test")</script>')
+        self.setup_bookmark(
+            notes='<b ld-fetch="https://example.com" ld-on="click">bold text</b>'
+        )
         html = self.render_template()
 
         note_html = '&lt;script&gt;alert("test")&lt;/script&gt;'
-        self.assertNotes(html, note_html, 1)
+        self.assertIn(note_html, html, 1)
+
+        note_html = "<b>bold text</b>"
+        self.assertIn(note_html, html, 1)
 
     def test_notes_are_hidden_initially_by_default(self):
         self.setup_bookmark(notes="Test note")

bookmarks/tests/test_bookmarks_service.py

@@ -1,10 +1,13 @@
+import os
+import tempfile
 from unittest.mock import patch
 
 from django.contrib.auth import get_user_model
-from django.test import TestCase
+from django.core.files.uploadedfile import SimpleUploadedFile
+from django.test import TestCase, override_settings
 from django.utils import timezone
 
-from bookmarks.models import Bookmark, Tag
+from bookmarks.models import Bookmark, BookmarkAsset, Tag
 from bookmarks.services import tasks
 from bookmarks.services import website_loader
 from bookmarks.services.bookmarks import (
@@ -21,6 +24,7 @@ from bookmarks.services.bookmarks import (
     mark_bookmarks_as_unread,
     share_bookmarks,
     unshare_bookmarks,
+    upload_asset,
 )
 from bookmarks.services.website_loader import WebsiteMetadata
 from bookmarks.tests.helpers import BookmarkFactoryMixin
@@ -38,7 +42,10 @@ class BookmarkServiceTestCase(TestCase, BookmarkFactoryMixin):
             website_loader, "load_website_metadata"
         ) as mock_load_website_metadata:
             expected_metadata = WebsiteMetadata(
-                "https://example.com", "Website title", "Website description"
+                "https://example.com",
+                "Website title",
+                "Website description",
+                "https://example.com/preview.png",
             )
             mock_load_website_metadata.return_value = expected_metadata
 
@@ -123,6 +130,18 @@ class BookmarkServiceTestCase(TestCase, BookmarkFactoryMixin):
 
             mock_create_html_snapshot.assert_not_called()
 
+    def test_create_should_add_tags_from_auto_tagging(self):
+        tag1 = self.setup_tag()
+        tag2 = self.setup_tag()
+        profile = self.get_or_create_test_user().profile
+        profile.auto_tagging_rules = f"example.com {tag2.name}"
+        profile.save()
+
+        bookmark_data = Bookmark(url="https://example.com")
+        bookmark = create_bookmark(bookmark_data, tag1.name, self.user)
+
+        self.assertCountEqual(bookmark.tags.all(), [tag1, tag2])
+
     def test_update_should_create_web_archive_snapshot_if_url_did_change(self):
         with patch.object(
             tasks, "create_web_archive_snapshot"
@@ -153,6 +172,7 @@ class BookmarkServiceTestCase(TestCase, BookmarkFactoryMixin):
                 "https://example.com/updated",
                 "Updated website title",
                 "Updated website description",
+                "https://example.com/preview.png",
             )
             mock_load_website_metadata.return_value = expected_metadata
 
@@ -193,6 +213,18 @@ class BookmarkServiceTestCase(TestCase, BookmarkFactoryMixin):
 
             mock_create_html_snapshot.assert_not_called()
 
+    def test_update_should_add_tags_from_auto_tagging(self):
+        tag1 = self.setup_tag()
+        tag2 = self.setup_tag()
+        profile = self.get_or_create_test_user().profile
+        profile.auto_tagging_rules = f"example.com {tag2.name}"
+        profile.save()
+        bookmark = self.setup_bookmark(url="https://example.com")
+
+        update_bookmark(bookmark, tag1.name, self.user)
+
+        self.assertCountEqual(bookmark.tags.all(), [tag1, tag2])
+
     def test_archive_bookmark(self):
         bookmark = Bookmark(
             url="https://example.com",
@@ -835,3 +867,50 @@ class BookmarkServiceTestCase(TestCase, BookmarkFactoryMixin):
         self.assertFalse(Bookmark.objects.get(id=bookmark1.id).shared)
         self.assertFalse(Bookmark.objects.get(id=bookmark2.id).shared)
         self.assertFalse(Bookmark.objects.get(id=bookmark3.id).shared)
+
+    def test_upload_asset_should_save_file(self):
+        bookmark = self.setup_bookmark()
+        with tempfile.TemporaryDirectory() as temp_assets:
+            with override_settings(LD_ASSET_FOLDER=temp_assets):
+                file_content = b"file content"
+                upload_file = SimpleUploadedFile(
+                    "test_file.txt", file_content, content_type="text/plain"
+                )
+                upload_asset(bookmark, upload_file)
+
+                assets = bookmark.bookmarkasset_set.all()
+                self.assertEqual(1, len(assets))
+
+                asset = assets[0]
+                self.assertEqual("test_file.txt", asset.display_name)
+                self.assertEqual("text/plain", asset.content_type)
+                self.assertEqual(upload_file.size, asset.file_size)
+                self.assertEqual(BookmarkAsset.STATUS_COMPLETE, asset.status)
+                self.assertTrue(asset.file.startswith("upload_"))
+                self.assertTrue(asset.file.endswith(upload_file.name))
+
+                # check file exists
+                filepath = os.path.join(temp_assets, asset.file)
+                self.assertTrue(os.path.exists(filepath))
+                with open(filepath, "rb") as f:
+                    self.assertEqual(file_content, f.read())
+
+    def test_upload_asset_should_be_failed_if_saving_file_fails(self):
+        bookmark = self.setup_bookmark()
+        # Use an invalid path to force an error
+        with override_settings(LD_ASSET_FOLDER="/non/existing/folder"):
+            file_content = b"file content"
+            upload_file = SimpleUploadedFile(
+                "test_file.txt", file_content, content_type="text/plain"
+            )
+            upload_asset(bookmark, upload_file)
+
+            assets = bookmark.bookmarkasset_set.all()
+            self.assertEqual(1, len(assets))
+
+            asset = assets[0]
+            self.assertEqual("test_file.txt", asset.display_name)
+            self.assertEqual("text/plain", asset.content_type)
+            self.assertIsNone(asset.file_size)
+            self.assertEqual(BookmarkAsset.STATUS_FAILURE, asset.status)
+            self.assertEqual("", asset.file)

bookmarks/tests/test_favicon_loader.py

@@ -1,8 +1,9 @@
 import io
 import os.path
 import time
+import tempfile
 from pathlib import Path
-from unittest import mock, skip
+from unittest import mock
 
 from django.conf import settings
 from django.test import TestCase, override_settings
@@ -29,17 +30,21 @@ class MockStreamingResponse:
 
 class FaviconLoaderTestCase(TestCase):
     def setUp(self) -> None:
-        self.ensure_favicon_folder()
-        self.clear_favicon_folder()
+        self.temp_favicon_folder = tempfile.TemporaryDirectory()
+        self.favicon_folder_override = self.settings(
+            LD_FAVICON_FOLDER=self.temp_favicon_folder.name
+        )
+        self.favicon_folder_override.enable()
+
+    def tearDown(self) -> None:
+        self.temp_favicon_folder.cleanup()
+        self.favicon_folder_override.disable()
 
     def create_mock_response(self, icon_data=mock_icon_data, content_type="image/png"):
         mock_response = mock.Mock()
         mock_response.raw = io.BytesIO(icon_data)
         return MockStreamingResponse(icon_data, content_type)
 
-    def ensure_favicon_folder(self):
-        Path(settings.LD_FAVICON_FOLDER).mkdir(parents=True, exist_ok=True)
-
     def clear_favicon_folder(self):
         folder = Path(settings.LD_FAVICON_FOLDER)
         for file in folder.iterdir():
@@ -177,7 +182,6 @@ class FaviconLoaderTestCase(TestCase):
             self.assertTrue(self.icon_exists("https_example_com.png"))
 
         self.clear_favicon_folder()
-        self.ensure_favicon_folder()
 
         with mock.patch("requests.get") as mock_get:
             mock_get.return_value = self.create_mock_response(

bookmarks/tests/test_importer.py

@@ -465,3 +465,14 @@ class ImporterTestCase(TestCase, BookmarkFactoryMixin, ImportTestMixin):
             import_netscape_html(test_html, user)
 
             mock_schedule_bookmarks_without_favicons.assert_called_once_with(user)
+
+    def test_schedule_preview_loading(self):
+        user = self.get_or_create_test_user()
+        test_html = self.render_html(tags_html="")
+
+        with patch.object(
+            tasks, "schedule_bookmarks_without_previews"
+        ) as mock_schedule_bookmarks_without_previews:
+            import_netscape_html(test_html, user)
+
+            mock_schedule_bookmarks_without_previews.assert_called_once_with(user)

bookmarks/tests/test_oidc_support.py

@@ -49,3 +49,15 @@ class OidcSupportTest(TestCase):
             base_settings.AUTHENTICATION_BACKENDS,
         )
         del os.environ["LD_ENABLE_OIDC"]  # Remove the temporary environment variable
+
+    def test_default_settings(self):
+        os.environ["LD_ENABLE_OIDC"] = "True"
+        base_settings = importlib.import_module("siteroot.settings.base")
+        importlib.reload(base_settings)
+
+        self.assertEqual(
+            True,
+            base_settings.OIDC_VERIFY_SSL,
+        )
+
+        del os.environ["LD_ENABLE_OIDC"]

bookmarks/tests/test_preview_image_loader.py

@@ -0,0 +1,205 @@
+import io
+import os
+import tempfile
+from pathlib import Path
+from unittest import mock
+
+from django.conf import settings
+from django.test import TestCase
+
+from bookmarks.services import preview_image_loader
+
+mock_image_data = b"mock_image"
+
+
+class MockStreamingResponse:
+    def __init__(
+        self,
+        url,
+        data=mock_image_data,
+        content_type="image/png",
+        content_length=None,
+        status_code=200,
+    ):
+        self.url = url
+        self.chunks = [data]
+        self.status_code = status_code
+        if not content_length:
+            content_length = len(data)
+        self.headers = {"Content-Type": content_type, "Content-Length": content_length}
+
+    def iter_content(self, **kwargs):
+        return self.chunks
+
+    def __enter__(self):
+        return self
+
+    def __exit__(self, exc_type, exc_value, traceback):
+        pass
+
+
+class PreviewImageLoaderTestCase(TestCase):
+    def setUp(self) -> None:
+        self.temp_folder = tempfile.TemporaryDirectory()
+        self.settings_override = self.settings(LD_PREVIEW_FOLDER=self.temp_folder.name)
+        self.settings_override.enable()
+        self.mock_load_website_metadata_patcher = mock.patch(
+            "bookmarks.services.website_loader.load_website_metadata"
+        )
+        self.mock_load_website_metadata = (
+            self.mock_load_website_metadata_patcher.start()
+        )
+        self.mock_load_website_metadata.return_value = mock.Mock(
+            preview_image="https://example.com/image.png"
+        )
+
+    def tearDown(self) -> None:
+        self.temp_folder.cleanup()
+        self.settings_override.disable()
+        self.mock_load_website_metadata_patcher.stop()
+
+    def create_mock_response(
+        self,
+        url="https://example.com/image.png",
+        icon_data=mock_image_data,
+        content_type="image/png",
+        content_length=len(mock_image_data),
+        status_code=200,
+    ):
+        mock_response = mock.Mock()
+        mock_response.raw = io.BytesIO(icon_data)
+        return MockStreamingResponse(
+            url, icon_data, content_type, content_length, status_code
+        )
+
+    def get_image_path(self, filename):
+        return Path(os.path.join(settings.LD_PREVIEW_FOLDER, filename))
+
+    def assertImageExists(self, filename, data):
+        self.assertTrue(self.get_image_path(filename).exists())
+        self.assertEqual(self.get_image_path(filename).read_bytes(), data)
+
+    def assertNoImageExists(self):
+        self.assertFalse(os.listdir(settings.LD_PREVIEW_FOLDER))
+
+    def test_load_preview_image(self):
+        with mock.patch("requests.get") as mock_get:
+            mock_get.return_value = self.create_mock_response()
+
+            file = preview_image_loader.load_preview_image("https://example.com")
+
+            self.assertIsNotNone(file)
+            self.assertImageExists(file, mock_image_data)
+
+    def test_load_preview_image_returns_none_if_no_preview_image_detected(self):
+        with mock.patch("requests.get") as mock_get:
+            mock_get.return_value = self.create_mock_response()
+            self.mock_load_website_metadata.return_value = mock.Mock(preview_image=None)
+
+            file = preview_image_loader.load_preview_image("https://example.com")
+
+            self.assertIsNone(file)
+            self.assertNoImageExists()
+
+    def test_load_preview_image_returns_none_for_invalid_status_code(self):
+        invalid_status_codes = [199, 300, 400, 500]
+
+        for status_code in invalid_status_codes:
+            with mock.patch("requests.get") as mock_get:
+                mock_get.return_value = self.create_mock_response(
+                    status_code=status_code
+                )
+
+                file = preview_image_loader.load_preview_image("https://example.com")
+
+                self.assertIsNone(file)
+                self.assertNoImageExists()
+
+    def test_load_preview_image_returns_none_if_content_length_exceeds_limit(self):
+        # exceeds max size
+        with mock.patch("requests.get") as mock_get:
+            mock_get.return_value = self.create_mock_response(
+                content_length=settings.LD_PREVIEW_MAX_SIZE + 1
+            )
+
+            file = preview_image_loader.load_preview_image("https://example.com")
+
+            self.assertIsNone(file)
+            self.assertNoImageExists()
+
+        # equals max size
+        with mock.patch("requests.get") as mock_get:
+            mock_get.return_value = self.create_mock_response(
+                content_length=settings.LD_PREVIEW_MAX_SIZE
+            )
+
+            file = preview_image_loader.load_preview_image("https://example.com")
+
+            self.assertIsNotNone(file)
+            self.assertImageExists(file, mock_image_data)
+
+    def test_load_preview_image_returns_none_for_invalid_content_type(self):
+        invalid_content_types = ["text/html", "application/json"]
+
+        for content_type in invalid_content_types:
+            with mock.patch("requests.get") as mock_get:
+                mock_get.return_value = self.create_mock_response(
+                    content_type=content_type
+                )
+
+                file = preview_image_loader.load_preview_image("https://example.com")
+
+                self.assertIsNone(file)
+                self.assertNoImageExists()
+
+        valid_content_types = ["image/png", "image/jpeg", "image/gif"]
+
+        for content_type in valid_content_types:
+            with mock.patch("requests.get") as mock_get:
+                mock_get.return_value = self.create_mock_response(
+                    content_type=content_type
+                )
+
+                file = preview_image_loader.load_preview_image("https://example.com")
+
+                self.assertIsNotNone(file)
+                self.assertImageExists(file, mock_image_data)
+
+    def test_load_preview_image_returns_none_if_download_exceeds_content_length(self):
+        with mock.patch("requests.get") as mock_get:
+            mock_get.return_value = self.create_mock_response(content_length=1)
+
+            file = preview_image_loader.load_preview_image("https://example.com")
+
+            self.assertIsNone(file)
+            self.assertNoImageExists()
+
+    def test_load_preview_image_creates_folder_if_not_exists(self):
+        with mock.patch("requests.get") as mock_get:
+            mock_get.return_value = self.create_mock_response()
+
+            folder = Path(settings.LD_PREVIEW_FOLDER)
+            folder.rmdir()
+
+            self.assertFalse(folder.exists())
+
+            preview_image_loader.load_preview_image("https://example.com")
+
+            self.assertTrue(folder.exists())
+
+    def test_guess_file_extension(self):
+        with mock.patch("requests.get") as mock_get:
+            mock_get.return_value = self.create_mock_response(content_type="image/png")
+
+            file = preview_image_loader.load_preview_image("https://example.com")
+
+            self.assertImageExists(file, mock_image_data)
+            self.assertEqual("png", file.split(".")[-1])
+
+        with mock.patch("requests.get") as mock_get:
+            mock_get.return_value = self.create_mock_response(content_type="image/jpeg")
+
+            file = preview_image_loader.load_preview_image("https://example.com")
+
+            self.assertImageExists(file, mock_image_data)
+            self.assertEqual("jpg", file.split(".")[-1])

bookmarks/tests/test_settings_general_view.py

@@ -31,8 +31,10 @@ class SettingsGeneralViewTestCase(TestCase, BookmarkFactoryMixin):
             "enable_sharing": False,
             "enable_public_sharing": False,
             "enable_favicons": False,
+            "enable_preview_images": False,
             "enable_automatic_html_snapshots": True,
             "tag_search": UserProfile.TAG_SEARCH_STRICT,
+            "tag_grouping": UserProfile.TAG_GROUPING_ALPHABETICAL,
             "display_url": False,
             "display_view_bookmark_action": True,
             "display_edit_bookmark_action": True,
@@ -40,10 +42,29 @@ class SettingsGeneralViewTestCase(TestCase, BookmarkFactoryMixin):
             "display_remove_bookmark_action": True,
             "permanent_notes": False,
             "custom_css": "",
+            "auto_tagging_rules": "",
         }
 
         return {**form_data, **overrides}
 
+    def assertSuccessMessage(self, html, message: str, count=1):
+        self.assertInHTML(
+            f"""
+            <div class="toast toast-success mb-4">{ message }</div>
+        """,
+            html,
+            count=count,
+        )
+
+    def assertErrorMessage(self, html, message: str, count=1):
+        self.assertInHTML(
+            f"""
+            <div class="toast toast-error mb-4">{ message }</div>
+        """,
+            html,
+            count=count,
+        )
+
     def test_should_render_successfully(self):
         response = self.client.get(reverse("bookmarks:settings.general"))
 
@@ -70,15 +91,19 @@ class SettingsGeneralViewTestCase(TestCase, BookmarkFactoryMixin):
             "enable_sharing": True,
             "enable_public_sharing": True,
             "enable_favicons": True,
+            "enable_preview_images": True,
             "enable_automatic_html_snapshots": False,
             "tag_search": UserProfile.TAG_SEARCH_LAX,
+            "tag_grouping": UserProfile.TAG_GROUPING_DISABLED,
             "display_url": True,
             "display_view_bookmark_action": False,
             "display_edit_bookmark_action": False,
             "display_archive_bookmark_action": False,
             "display_remove_bookmark_action": False,
             "permanent_notes": True,
+            "default_mark_unread": True,
             "custom_css": "body { background-color: #000; }",
+            "auto_tagging_rules": "example.com tag",
         }
         response = self.client.post(reverse("bookmarks:settings.general"), form_data)
         html = response.content.decode()
@@ -112,11 +137,15 @@ class SettingsGeneralViewTestCase(TestCase, BookmarkFactoryMixin):
         self.assertEqual(
             self.user.profile.enable_favicons, form_data["enable_favicons"]
         )
+        self.assertEqual(
+            self.user.profile.enable_preview_images, form_data["enable_preview_images"]
+        )
         self.assertEqual(
             self.user.profile.enable_automatic_html_snapshots,
             form_data["enable_automatic_html_snapshots"],
         )
         self.assertEqual(self.user.profile.tag_search, form_data["tag_search"])
+        self.assertEqual(self.user.profile.tag_grouping, form_data["tag_grouping"])
         self.assertEqual(self.user.profile.display_url, form_data["display_url"])
         self.assertEqual(
             self.user.profile.display_view_bookmark_action,
@@ -137,13 +166,14 @@ class SettingsGeneralViewTestCase(TestCase, BookmarkFactoryMixin):
         self.assertEqual(
             self.user.profile.permanent_notes, form_data["permanent_notes"]
         )
-        self.assertEqual(self.user.profile.custom_css, form_data["custom_css"])
-        self.assertInHTML(
-            """
-                <p class="form-input-hint">Profile updated</p>
-            """,
-            html,
+        self.assertEqual(
+            self.user.profile.default_mark_unread, form_data["default_mark_unread"]
         )
+        self.assertEqual(self.user.profile.custom_css, form_data["custom_css"])
+        self.assertEqual(
+            self.user.profile.auto_tagging_rules, form_data["auto_tagging_rules"]
+        )
+        self.assertSuccessMessage(html, "Profile updated")
 
     def test_update_profile_should_not_be_called_without_respective_form_action(self):
         form_data = {
@@ -156,13 +186,7 @@ class SettingsGeneralViewTestCase(TestCase, BookmarkFactoryMixin):
 
         self.assertEqual(response.status_code, 200)
         self.assertEqual(self.user.profile.theme, UserProfile.THEME_AUTO)
-        self.assertInHTML(
-            """
-                <p class="form-input-hint">Profile updated</p>
-            """,
-            html,
-            count=0,
-        )
+        self.assertSuccessMessage(html, "Profile updated", count=0)
 
     def test_enable_favicons_should_schedule_icon_update(self):
         with patch.object(
@@ -210,13 +234,8 @@ class SettingsGeneralViewTestCase(TestCase, BookmarkFactoryMixin):
             html = response.content.decode()
 
             mock_schedule_refresh_favicons.assert_called_once()
-            self.assertInHTML(
-                """
-                <p class="form-input-hint">
-                    Scheduled favicon update. This may take a while...
-                </p>
-            """,
-                html,
+            self.assertSuccessMessage(
+                html, "Scheduled favicon update. This may take a while..."
             )
 
     def test_refresh_favicons_should_not_be_called_without_respective_form_action(self):
@@ -230,14 +249,8 @@ class SettingsGeneralViewTestCase(TestCase, BookmarkFactoryMixin):
             html = response.content.decode()
 
             mock_schedule_refresh_favicons.assert_not_called()
-            self.assertInHTML(
-                """
-                <p class="form-input-hint">
-                    Scheduled favicon update. This may take a while...
-                </p>
-            """,
-                html,
-                count=0,
+            self.assertSuccessMessage(
+                html, "Scheduled favicon update. This may take a while...", count=0
             )
 
     def test_refresh_favicons_should_be_visible_when_favicons_enabled_in_profile(self):
@@ -291,6 +304,39 @@ class SettingsGeneralViewTestCase(TestCase, BookmarkFactoryMixin):
             count=0,
         )
 
+    def test_enable_preview_image_should_schedule_preview_update(self):
+        with patch.object(
+            tasks, "schedule_bookmarks_without_previews"
+        ) as mock_schedule_bookmarks_without_previews:
+            # Enabling favicons schedules update
+            form_data = self.create_profile_form_data(
+                {
+                    "update_profile": "",
+                    "enable_preview_images": True,
+                }
+            )
+            self.client.post(reverse("bookmarks:settings.general"), form_data)
+
+            mock_schedule_bookmarks_without_previews.assert_called_once_with(self.user)
+
+            # No update scheduled if favicons are already enabled
+            mock_schedule_bookmarks_without_previews.reset_mock()
+
+            self.client.post(reverse("bookmarks:settings.general"), form_data)
+
+            mock_schedule_bookmarks_without_previews.assert_not_called()
+
+            # No update scheduled when disabling favicons
+            form_data = self.create_profile_form_data(
+                {
+                    "enable_preview_images": False,
+                }
+            )
+
+            self.client.post(reverse("bookmarks:settings.general"), form_data)
+
+            mock_schedule_bookmarks_without_previews.assert_not_called()
+
     def test_automatic_html_snapshots_should_be_hidden_when_snapshots_not_supported(
         self,
     ):
@@ -365,3 +411,57 @@ class SettingsGeneralViewTestCase(TestCase, BookmarkFactoryMixin):
         with patch.object(requests, "get", return_value=latest_version_response_mock):
             version_info = get_version_info(random.random())
             self.assertEqual(version_info, app_version)
+
+    @override_settings(LD_ENABLE_SNAPSHOTS=True)
+    def test_create_missing_html_snapshots(self):
+        with patch.object(
+            tasks, "create_missing_html_snapshots"
+        ) as mock_create_missing_html_snapshots:
+            mock_create_missing_html_snapshots.return_value = 5
+            form_data = {
+                "create_missing_html_snapshots": "",
+            }
+            response = self.client.post(
+                reverse("bookmarks:settings.general"), form_data
+            )
+            html = response.content.decode()
+
+            mock_create_missing_html_snapshots.assert_called_once()
+            self.assertSuccessMessage(
+                html, "Queued 5 missing snapshots. This may take a while..."
+            )
+
+    @override_settings(LD_ENABLE_SNAPSHOTS=True)
+    def test_create_missing_html_snapshots_no_missing_snapshots(self):
+        with patch.object(
+            tasks, "create_missing_html_snapshots"
+        ) as mock_create_missing_html_snapshots:
+            mock_create_missing_html_snapshots.return_value = 0
+            form_data = {
+                "create_missing_html_snapshots": "",
+            }
+            response = self.client.post(
+                reverse("bookmarks:settings.general"), form_data
+            )
+            html = response.content.decode()
+
+            mock_create_missing_html_snapshots.assert_called_once()
+            self.assertSuccessMessage(html, "No missing snapshots found.")
+
+    def test_create_missing_html_snapshots_should_not_be_called_without_respective_form_action(
+        self,
+    ):
+        with patch.object(
+            tasks, "create_missing_html_snapshots"
+        ) as mock_create_missing_html_snapshots:
+            mock_create_missing_html_snapshots.return_value = 5
+            form_data = {}
+            response = self.client.post(
+                reverse("bookmarks:settings.general"), form_data
+            )
+            html = response.content.decode()
+
+            mock_create_missing_html_snapshots.assert_not_called()
+            self.assertSuccessMessage(
+                html, "Queued 5 missing snapshots. This may take a while...", count=0
+            )

bookmarks/tests/test_settings_import_view.py

@@ -11,19 +11,27 @@ class SettingsImportViewTestCase(TestCase, BookmarkFactoryMixin):
         user = self.get_or_create_test_user()
         self.client.force_login(user)
 
-    def assertFormSuccessHint(self, response, text: str):
-        self.assertContains(response, '<div class="has-success">')
-        self.assertContains(response, text)
+    def assertSuccessMessage(self, response, message: str):
+        self.assertInHTML(
+            f"""
+            <div class="toast toast-success mb-4">{ message }</div>
+        """,
+            response.content.decode("utf-8"),
+        )
 
-    def assertNoFormSuccessHint(self, response):
-        self.assertNotContains(response, '<div class="has-success">')
+    def assertNoSuccessMessage(self, response):
+        self.assertNotContains(response, '<div class="toast toast-success mb-4">')
 
-    def assertFormErrorHint(self, response, text: str):
-        self.assertContains(response, '<div class="has-error">')
-        self.assertContains(response, text)
+    def assertErrorMessage(self, response, message: str):
+        self.assertInHTML(
+            f"""
+            <div class="toast toast-error mb-4">{ message }</div>
+        """,
+            response.content.decode("utf-8"),
+        )
 
-    def assertNoFormErrorHint(self, response):
-        self.assertNotContains(response, '<div class="has-error">')
+    def assertNoErrorMessage(self, response):
+        self.assertNotContains(response, '<div class="toast toast-error mb-4">')
 
     def test_should_import_successfully(self):
         with open(
@@ -36,10 +44,10 @@ class SettingsImportViewTestCase(TestCase, BookmarkFactoryMixin):
             )
 
             self.assertRedirects(response, reverse("bookmarks:settings.general"))
-            self.assertFormSuccessHint(
-                response, "3 bookmarks were successfully imported"
+            self.assertSuccessMessage(
+                response, "3 bookmarks were successfully imported."
             )
-            self.assertNoFormErrorHint(response)
+            self.assertNoErrorMessage(response)
 
     def test_should_check_authentication(self):
         self.client.logout()
@@ -53,8 +61,8 @@ class SettingsImportViewTestCase(TestCase, BookmarkFactoryMixin):
         response = self.client.post(reverse("bookmarks:settings.import"), follow=True)
 
         self.assertRedirects(response, reverse("bookmarks:settings.general"))
-        self.assertNoFormSuccessHint(response)
-        self.assertFormErrorHint(response, "Please select a file to import.")
+        self.assertNoSuccessMessage(response)
+        self.assertErrorMessage(response, "Please select a file to import.")
 
     @disable_logging
     def test_should_show_hint_if_import_raises_exception(self):
@@ -68,8 +76,8 @@ class SettingsImportViewTestCase(TestCase, BookmarkFactoryMixin):
             )
 
             self.assertRedirects(response, reverse("bookmarks:settings.general"))
-            self.assertNoFormSuccessHint(response)
-            self.assertFormErrorHint(
+            self.assertNoSuccessMessage(response)
+            self.assertErrorMessage(
                 response, "An error occurred during bookmark import."
             )
 
@@ -87,10 +95,13 @@ class SettingsImportViewTestCase(TestCase, BookmarkFactoryMixin):
             )
 
             self.assertRedirects(response, reverse("bookmarks:settings.general"))
-            self.assertFormSuccessHint(
-                response, "2 bookmarks were successfully imported"
+            self.assertSuccessMessage(
+                response, "2 bookmarks were successfully imported."
+            )
+            self.assertErrorMessage(
+                response,
+                "1 bookmarks could not be imported. Please check the logs for more details.",
             )
-            self.assertFormErrorHint(response, "1 bookmarks could not be imported")
 
     def test_should_respect_map_private_flag_option(self):
         with open(

bookmarks/tests/test_singlefile_service.py

@@ -1,9 +1,9 @@
 import gzip
 import os
-from unittest import mock
 import subprocess
+from unittest import mock
 
-from django.test import TestCase
+from django.test import TestCase, override_settings
 
 from bookmarks.services import singlefile
 
@@ -24,9 +24,11 @@ class SingleFileServiceTestCase(TestCase):
             file.write(self.html_content)
 
     def test_create_snapshot(self):
-        with mock.patch("subprocess.run") as mock_run:
-            mock_run.side_effect = self.create_test_file
+        mock_process = mock.Mock()
+        mock_process.wait.return_value = 0
+        self.create_test_file()
 
+        with mock.patch("subprocess.Popen", return_value=mock_process):
             singlefile.create_snapshot("http://example.com", self.html_filepath)
 
             self.assertTrue(os.path.exists(self.html_filepath))
@@ -38,13 +40,80 @@ class SingleFileServiceTestCase(TestCase):
 
     def test_create_snapshot_failure(self):
         # subprocess fails - which it probably doesn't as single-file doesn't return exit codes
-        with mock.patch("subprocess.run") as mock_run:
-            mock_run.side_effect = subprocess.CalledProcessError(1, "command")
+        with mock.patch("subprocess.Popen") as mock_popen:
+            mock_popen.side_effect = subprocess.CalledProcessError(1, "command")
 
             with self.assertRaises(singlefile.SingeFileError):
                 singlefile.create_snapshot("http://example.com", self.html_filepath)
 
         # so also check that it raises error if output file isn't created
-        with mock.patch("subprocess.run") as mock_run:
+        with mock.patch("subprocess.Popen"):
             with self.assertRaises(singlefile.SingeFileError):
                 singlefile.create_snapshot("http://example.com", self.html_filepath)
+
+    def test_create_snapshot_empty_options(self):
+        mock_process = mock.Mock()
+        mock_process.wait.return_value = 0
+        self.create_test_file()
+
+        with mock.patch("subprocess.Popen") as mock_popen:
+            singlefile.create_snapshot("http://example.com", self.html_filepath)
+
+            expected_args = [
+                "single-file",
+                '--browser-arg="--headless=new"',
+                '--browser-arg="--user-data-dir=./chromium-profile"',
+                '--browser-arg="--no-sandbox"',
+                '--browser-arg="--load-extension=uBlock0.chromium"',
+                "http://example.com",
+                self.html_filepath + ".tmp",
+            ]
+            mock_popen.assert_called_with(expected_args, start_new_session=True)
+
+    @override_settings(
+        LD_SINGLEFILE_OPTIONS='--some-option "some value" --another-option "another value" --third-option="third value"'
+    )
+    def test_create_snapshot_custom_options(self):
+        mock_process = mock.Mock()
+        mock_process.wait.return_value = 0
+        self.create_test_file()
+
+        with mock.patch("subprocess.Popen") as mock_popen:
+            singlefile.create_snapshot("http://example.com", self.html_filepath)
+
+            expected_args = [
+                "single-file",
+                '--browser-arg="--headless=new"',
+                '--browser-arg="--user-data-dir=./chromium-profile"',
+                '--browser-arg="--no-sandbox"',
+                '--browser-arg="--load-extension=uBlock0.chromium"',
+                "--some-option",
+                "some value",
+                "--another-option",
+                "another value",
+                "--third-option=third value",
+                "http://example.com",
+                self.html_filepath + ".tmp",
+            ]
+            mock_popen.assert_called_with(expected_args, start_new_session=True)
+
+    def test_create_snapshot_default_timeout_setting(self):
+        mock_process = mock.Mock()
+        mock_process.wait.return_value = 0
+        self.create_test_file()
+
+        with mock.patch("subprocess.Popen", return_value=mock_process):
+            singlefile.create_snapshot("http://example.com", self.html_filepath)
+
+            mock_process.wait.assert_called_with(timeout=120)
+
+    @override_settings(LD_SINGLEFILE_TIMEOUT_SEC=180)
+    def test_create_snapshot_custom_timeout_setting(self):
+        mock_process = mock.Mock()
+        mock_process.wait.return_value = 0
+        self.create_test_file()
+
+        with mock.patch("subprocess.Popen", return_value=mock_process):
+            singlefile.create_snapshot("http://example.com", self.html_filepath)
+
+            mock_process.wait.assert_called_with(timeout=180)

bookmarks/tests/test_tag_cloud_template.py

@@ -140,6 +140,43 @@ class TagCloudTemplateTest(TestCase, BookmarkFactoryMixin, HtmlTestMixin):
             ],
         )
 
+    def test_group_when_grouping_disabled(self):
+        profile = self.get_or_create_test_user().profile
+        profile.tag_grouping = UserProfile.TAG_GROUPING_DISABLED
+        profile.save()
+
+        tags = [
+            self.setup_tag(name="Cockatoo"),
+            self.setup_tag(name="Badger"),
+            self.setup_tag(name="Buffalo"),
+            self.setup_tag(name="Chihuahua"),
+            self.setup_tag(name="Alpaca"),
+            self.setup_tag(name="Coyote"),
+            self.setup_tag(name="Aardvark"),
+            self.setup_tag(name="Bumblebee"),
+            self.setup_tag(name="Armadillo"),
+        ]
+        self.setup_bookmark(tags=tags)
+
+        rendered_template = self.render_template()
+
+        self.assertTagGroups(
+            rendered_template,
+            [
+                [
+                    "Aardvark",
+                    "Alpaca",
+                    "Armadillo",
+                    "Badger",
+                    "Buffalo",
+                    "Bumblebee",
+                    "Chihuahua",
+                    "Cockatoo",
+                    "Coyote",
+                ],
+            ],
+        )
+
     def test_no_duplicate_tag_names(self):
         tags = [
             self.setup_tag(name="shared", user=self.setup_user(enable_sharing=True)),

bookmarks/tests/test_website_loader.py

@@ -29,7 +29,9 @@ class WebsiteLoaderTestCase(TestCase):
         # clear cached metadata before test run
         website_loader.load_website_metadata.cache_clear()
 
-    def render_html_document(self, title, description="", og_description=""):
+    def render_html_document(
+        self, title, description="", og_description="", og_image=""
+    ):
         meta_description = (
             f'<meta name="description" content="{description}">' if description else ""
         )
@@ -38,6 +40,9 @@ class WebsiteLoaderTestCase(TestCase):
             if og_description
             else ""
         )
+        meta_og_image = (
+            f'<meta property="og:image" content="{og_image}">' if og_image else ""
+        )
         return f"""
         <!DOCTYPE html>
         <html lang="en">
@@ -46,6 +51,7 @@ class WebsiteLoaderTestCase(TestCase):
             <title>{title}</title>
             {meta_description}
             {meta_og_description}
+            {meta_og_image}
         </head>
         <body></body>
         </html>
@@ -105,6 +111,7 @@ class WebsiteLoaderTestCase(TestCase):
             metadata = website_loader.load_website_metadata("https://example.com")
             self.assertEqual("test title", metadata.title)
             self.assertEqual("test description", metadata.description)
+            self.assertIsNone(metadata.preview_image)
 
     def test_load_website_metadata_trims_title_and_description(self):
         with mock.patch(
@@ -128,6 +135,44 @@ class WebsiteLoaderTestCase(TestCase):
             self.assertEqual("test title", metadata.title)
             self.assertEqual("test og description", metadata.description)
 
+    def test_load_website_metadata_using_og_image(self):
+        with mock.patch(
+            "bookmarks.services.website_loader.load_page"
+        ) as mock_load_page:
+            mock_load_page.return_value = self.render_html_document(
+                "test title", og_image="http://example.com/image.jpg"
+            )
+            metadata = website_loader.load_website_metadata("https://example.com")
+            self.assertEqual("http://example.com/image.jpg", metadata.preview_image)
+
+    def test_load_website_metadata_gets_absolute_og_image_path_when_path_starts_with_dots(
+        self,
+    ):
+        with mock.patch(
+            "bookmarks.services.website_loader.load_page"
+        ) as mock_load_page:
+            mock_load_page.return_value = self.render_html_document(
+                "test title", og_image="../image.jpg"
+            )
+            metadata = website_loader.load_website_metadata(
+                "https://example.com/a/b/page.html"
+            )
+            self.assertEqual("https://example.com/a/image.jpg", metadata.preview_image)
+
+    def test_load_website_metadata_gets_absolute_og_image_path_when_path_starts_with_slash(
+        self,
+    ):
+        with mock.patch(
+            "bookmarks.services.website_loader.load_page"
+        ) as mock_load_page:
+            mock_load_page.return_value = self.render_html_document(
+                "test title", og_image="/image.jpg"
+            )
+            metadata = website_loader.load_website_metadata(
+                "https://example.com/a/b/page.html"
+            )
+            self.assertEqual("https://example.com/image.jpg", metadata.preview_image)
+
     def test_load_website_metadata_prefers_description_over_og_description(self):
         with mock.patch(
             "bookmarks.services.website_loader.load_page"

bookmarks/urls.py

@@ -44,12 +44,22 @@ urlpatterns = [
         views.bookmarks.details_modal,
         name="details_modal",
     ),
+    path(
+        "bookmarks/<int:bookmark_id>/details_assets",
+        views.bookmarks.details_assets,
+        name="details_assets",
+    ),
     # Assets
     path(
         "assets/<int:asset_id>",
         views.assets.view,
         name="assets.view",
     ),
+    path(
+        "assets/<int:asset_id>/read",
+        views.assets.read,
+        name="assets.read",
+    ),
     # Partials
     path(
         "bookmarks/partials/bookmark-list/active",
@@ -61,6 +71,11 @@ urlpatterns = [
         partials.active_tag_cloud,
         name="partials.tag_cloud.active",
     ),
+    path(
+        "bookmarks/partials/tag-modal/active",
+        partials.active_tag_modal,
+        name="partials.tag_modal.active",
+    ),
     path(
         "bookmarks/partials/bookmark-list/archived",
         partials.archived_bookmark_list,
@@ -71,6 +86,11 @@ urlpatterns = [
         partials.archived_tag_cloud,
         name="partials.tag_cloud.archived",
     ),
+    path(
+        "bookmarks/partials/tag-modal/archived",
+        partials.archived_tag_modal,
+        name="partials.tag_modal.archived",
+    ),
     path(
         "bookmarks/partials/bookmark-list/shared",
         partials.shared_bookmark_list,
@@ -82,9 +102,9 @@ urlpatterns = [
         name="partials.tag_cloud.shared",
     ),
     path(
-        "bookmarks/partials/details-form/<int:bookmark_id>",
-        partials.details_form,
-        name="partials.details_form",
+        "bookmarks/partials/tag-modal/shared",
+        partials.shared_tag_modal,
+        name="partials.tag_modal.shared",
     ),
     # Settings
     path("settings", views.settings.general, name="settings.index"),

bookmarks/views/assets.py

@@ -6,11 +6,12 @@ from django.http import (
     HttpResponse,
     Http404,
 )
+from django.shortcuts import render
 
 from bookmarks.models import BookmarkAsset
 
 
-def view(request, asset_id: int):
+def _access_asset(request, asset_id: int):
     try:
         asset = BookmarkAsset.objects.get(pk=asset_id)
     except BookmarkAsset.DoesNotExist:
@@ -28,6 +29,10 @@ def view(request, asset_id: int):
     if not is_owner and not is_shared and not is_public_shared:
         raise Http404("Bookmark does not exist")
 
+    return asset
+
+
+def _get_asset_content(asset):
     filepath = os.path.join(settings.LD_ASSET_FOLDER, asset.file)
 
     if not os.path.exists(filepath):
@@ -40,4 +45,25 @@ def view(request, asset_id: int):
         with open(filepath, "rb") as f:
             content = f.read()
 
+    return content
+
+
+def view(request, asset_id: int):
+    asset = _access_asset(request, asset_id)
+    content = _get_asset_content(asset)
+
     return HttpResponse(content, content_type=asset.content_type)
+
+
+def read(request, asset_id: int):
+    asset = _access_asset(request, asset_id)
+    content = _get_asset_content(asset)
+    content = content.decode("utf-8")
+
+    return render(
+        request,
+        "bookmarks/read.html",
+        {
+            "content": content,
+        },
+    )

bookmarks/views/bookmarks.py

@@ -34,7 +34,7 @@ from bookmarks.services.bookmarks import (
     share_bookmarks,
     unshare_bookmarks,
 )
-from bookmarks.services import tasks
+from bookmarks.services import bookmarks as bookmark_actions, tasks
 from bookmarks.utils import get_safe_return_url
 from bookmarks.views.partials import contexts
 
@@ -145,6 +145,11 @@ def _details(request, bookmark_id: int, template: str):
             asset.delete()
         if "create_snapshot" in request.POST:
             tasks.create_html_snapshot(bookmark)
+        if "upload_asset" in request.POST:
+            file = request.FILES.get("upload_asset_file")
+            if not file:
+                return HttpResponseBadRequest("No file uploaded")
+            bookmark_actions.upload_asset(bookmark, file)
         else:
             bookmark.is_archived = request.POST.get("is_archived") == "on"
             bookmark.unread = request.POST.get("unread") == "on"
@@ -172,6 +177,10 @@ def details_modal(request, bookmark_id: int):
     return _details(request, bookmark_id, "bookmarks/details_modal.html")
 
 
+def details_assets(request, bookmark_id: int):
+    return _details(request, bookmark_id, "bookmarks/details/assets.html")
+
+
 def convert_tag_string(tag_string: str):
     # Tag strings coming from inputs are space-separated, however services.bookmarks functions expect comma-separated
     # strings
@@ -184,6 +193,7 @@ def new(request):
     initial_title = request.GET.get("title")
     initial_description = request.GET.get("description")
     initial_auto_close = "auto_close" in request.GET
+    initial_mark_unread = request.user.profile.default_mark_unread
 
     if request.method == "POST":
         form = BookmarkForm(request.POST)
@@ -206,6 +216,8 @@ def new(request):
             form.initial["description"] = initial_description
         if initial_auto_close:
             form.initial["auto_close"] = "true"
+        if initial_mark_unread:
+            form.initial["unread"] = "true"
 
     context = {
         "form": form,

bookmarks/views/partials/__init__.py

@@ -1,8 +1,6 @@
 from django.contrib.auth.decorators import login_required
-from django.http import Http404
 from django.shortcuts import render
 
-from bookmarks.models import Bookmark
 from bookmarks.views.partials import contexts
 
 
@@ -24,6 +22,13 @@ def active_tag_cloud(request):
     return render(request, "bookmarks/tag_cloud.html", {"tag_cloud": tag_cloud_context})
 
 
+@login_required
+def active_tag_modal(request):
+    tag_cloud_context = contexts.ActiveTagCloudContext(request)
+
+    return render(request, "bookmarks/tag_modal.html", {"tag_cloud": tag_cloud_context})
+
+
 @login_required
 def archived_bookmark_list(request):
     bookmark_list_context = contexts.ArchivedBookmarkListContext(request)
@@ -43,6 +48,12 @@ def archived_tag_cloud(request):
 
 
 @login_required
+def archived_tag_modal(request):
+    tag_cloud_context = contexts.ArchivedTagCloudContext(request)
+
+    return render(request, "bookmarks/tag_modal.html", {"tag_cloud": tag_cloud_context})
+
+
 def shared_bookmark_list(request):
     bookmark_list_context = contexts.SharedBookmarkListContext(request)
 
@@ -53,20 +64,13 @@ def shared_bookmark_list(request):
     )
 
 
-@login_required
 def shared_tag_cloud(request):
     tag_cloud_context = contexts.SharedTagCloudContext(request)
 
     return render(request, "bookmarks/tag_cloud.html", {"tag_cloud": tag_cloud_context})
 
 
-@login_required
-def details_form(request, bookmark_id: int):
-    try:
-        bookmark = Bookmark.objects.get(pk=bookmark_id, owner=request.user)
-    except Bookmark.DoesNotExist:
-        raise Http404("Bookmark does not exist")
+def shared_tag_modal(request):
+    tag_cloud_context = contexts.SharedTagCloudContext(request)
 
-    details_context = contexts.BookmarkDetailsContext(request, bookmark)
-
-    return render(request, "bookmarks/details/form.html", {"details": details_context})
+    return render(request, "bookmarks/tag_modal.html", {"tag_cloud": tag_cloud_context})

bookmarks/views/partials/contexts.py

@@ -23,6 +23,113 @@ DEFAULT_PAGE_SIZE = 30
 CJK_RE = re.compile(r"[\u4e00-\u9fff]+")
 
 
+class RequestContext:
+    index_view = "bookmarks:index"
+    action_view = "bookmarks:index.action"
+    bookmark_list_partial_view = "bookmarks:partials.bookmark_list.active"
+    tag_cloud_partial_view = "bookmarks:partials.tag_cloud.active"
+    tag_modal_partial_view = "bookmarks:partials.tag_modal.active"
+
+    def __init__(self, request: WSGIRequest):
+        self.request = request
+        self.index_url = reverse(self.index_view)
+        self.action_url = reverse(self.action_view)
+        self.bookmark_list_partial_url = reverse(self.bookmark_list_partial_view)
+        self.tag_cloud_partial_url = reverse(self.tag_cloud_partial_view)
+        self.tag_modal_partial_url = reverse(self.tag_modal_partial_view)
+        self.query_params = request.GET.copy()
+        self.query_params.pop("details", None)
+
+    def get_url(self, view_url: str, add: dict = None, remove: dict = None) -> str:
+        query_params = self.query_params.copy()
+        if add:
+            query_params.update(add)
+        if remove:
+            for key in remove:
+                query_params.pop(key, None)
+        encoded_params = query_params.urlencode()
+        return view_url + "?" + encoded_params if encoded_params else view_url
+
+    def index(self) -> str:
+        return self.get_url(self.index_url)
+
+    def action(self, return_url: str) -> str:
+        return self.get_url(self.action_url, add={"return_url": return_url})
+
+    def bookmark_list_partial(self) -> str:
+        return self.get_url(self.bookmark_list_partial_url)
+
+    def tag_cloud_partial(self) -> str:
+        return self.get_url(self.tag_cloud_partial_url)
+
+    def tag_modal_partial(self) -> str:
+        return self.get_url(self.tag_modal_partial_url)
+
+    def get_bookmark_query_set(self, search: BookmarkSearch):
+        raise Exception("Must be implemented by subclass")
+
+    def get_tag_query_set(self, search: BookmarkSearch):
+        raise Exception("Must be implemented by subclass")
+
+
+class ActiveBookmarksContext(RequestContext):
+    index_view = "bookmarks:index"
+    action_view = "bookmarks:index.action"
+    bookmark_list_partial_view = "bookmarks:partials.bookmark_list.active"
+    tag_cloud_partial_view = "bookmarks:partials.tag_cloud.active"
+    tag_modal_partial_view = "bookmarks:partials.tag_modal.active"
+
+    def get_bookmark_query_set(self, search: BookmarkSearch):
+        return queries.query_bookmarks(
+            self.request.user, self.request.user_profile, search
+        )
+
+    def get_tag_query_set(self, search: BookmarkSearch):
+        return queries.query_bookmark_tags(
+            self.request.user, self.request.user_profile, search
+        )
+
+
+class ArchivedBookmarksContext(RequestContext):
+    index_view = "bookmarks:archived"
+    action_view = "bookmarks:archived.action"
+    bookmark_list_partial_view = "bookmarks:partials.bookmark_list.archived"
+    tag_cloud_partial_view = "bookmarks:partials.tag_cloud.archived"
+    tag_modal_partial_view = "bookmarks:partials.tag_modal.archived"
+
+    def get_bookmark_query_set(self, search: BookmarkSearch):
+        return queries.query_archived_bookmarks(
+            self.request.user, self.request.user_profile, search
+        )
+
+    def get_tag_query_set(self, search: BookmarkSearch):
+        return queries.query_archived_bookmark_tags(
+            self.request.user, self.request.user_profile, search
+        )
+
+
+class SharedBookmarksContext(RequestContext):
+    index_view = "bookmarks:shared"
+    action_view = "bookmarks:shared.action"
+    bookmark_list_partial_view = "bookmarks:partials.bookmark_list.shared"
+    tag_cloud_partial_view = "bookmarks:partials.tag_cloud.shared"
+    tag_modal_partial_view = "bookmarks:partials.tag_modal.shared"
+
+    def get_bookmark_query_set(self, search: BookmarkSearch):
+        user = User.objects.filter(username=search.user).first()
+        public_only = not self.request.user.is_authenticated
+        return queries.query_shared_bookmarks(
+            user, self.request.user_profile, search, public_only
+        )
+
+    def get_tag_query_set(self, search: BookmarkSearch):
+        user = User.objects.filter(username=search.user).first()
+        public_only = not self.request.user.is_authenticated
+        return queries.query_shared_bookmark_tags(
+            user, self.request.user_profile, search, public_only
+        )
+
+
 class BookmarkItem:
     def __init__(self, bookmark: Bookmark, user: User, profile: UserProfile) -> None:
         self.bookmark = bookmark
@@ -38,6 +145,7 @@ class BookmarkItem:
         self.tag_names = bookmark.tag_names
         self.web_archive_snapshot_url = bookmark.web_archive_snapshot_url
         self.favicon_file = bookmark.favicon_file
+        self.preview_image_file = bookmark.preview_image_file
         self.is_archived = bookmark.is_archived
         self.unread = bookmark.unread
         self.owner = bookmark.owner
@@ -67,7 +175,10 @@ class BookmarkItem:
 
 
 class BookmarkListContext:
+    request_context = RequestContext
+
     def __init__(self, request: WSGIRequest) -> None:
+        request_context = self.request_context(request)
         user = request.user
         user_profile = request.user_profile
 
@@ -76,7 +187,7 @@ class BookmarkListContext:
             self.request.GET, user_profile.search_preferences
         )
 
-        query_set = self.get_bookmark_query_set()
+        query_set = request_context.get_bookmark_query_set(self.search)
         page_number = request.GET.get("page")
         paginator = Paginator(query_set, DEFAULT_PAGE_SIZE)
         bookmarks_page = paginator.get_page(page_number)
@@ -86,16 +197,15 @@ class BookmarkListContext:
         self.items = [
             BookmarkItem(bookmark, user, user_profile) for bookmark in bookmarks_page
         ]
-
         self.is_empty = paginator.count == 0
         self.bookmarks_page = bookmarks_page
         self.bookmarks_total = paginator.count
-        self.return_url = self.generate_return_url(
-            self.search, self.get_base_url(), page_number
-        )
-        self.action_url = self.generate_action_url(
-            self.search, self.get_base_action_url(), self.return_url
-        )
+
+        self.return_url = request_context.index()
+        self.action_url = request_context.action(return_url=self.return_url)
+        self.refresh_url = request_context.bookmark_list_partial()
+        self.tag_modal_url = request_context.tag_modal_partial()
+
         self.link_target = user_profile.bookmark_link_target
         self.date_display = user_profile.bookmark_date_display
         self.description_display = user_profile.bookmark_description_display
@@ -106,6 +216,7 @@ class BookmarkListContext:
         self.show_archive_action = user_profile.display_archive_bookmark_action
         self.show_remove_action = user_profile.display_remove_bookmark_action
         self.show_favicons = user_profile.enable_favicons
+        self.show_preview_images = user_profile.enable_preview_images
         self.show_notes = user_profile.permanent_notes
 
     @staticmethod
@@ -131,55 +242,17 @@ class BookmarkListContext:
             else base_action_url + "?" + query_string
         )
 
-    def get_base_url(self):
-        raise Exception("Must be implemented by subclass")
-
-    def get_base_action_url(self):
-        raise Exception("Must be implemented by subclass")
-
-    def get_bookmark_query_set(self):
-        raise Exception("Must be implemented by subclass")
-
 
 class ActiveBookmarkListContext(BookmarkListContext):
-    def get_base_url(self):
-        return reverse("bookmarks:index")
-
-    def get_base_action_url(self):
-        return reverse("bookmarks:index.action")
-
-    def get_bookmark_query_set(self):
-        return queries.query_bookmarks(
-            self.request.user, self.request.user_profile, self.search
-        )
+    request_context = ActiveBookmarksContext
 
 
 class ArchivedBookmarkListContext(BookmarkListContext):
-    def get_base_url(self):
-        return reverse("bookmarks:archived")
-
-    def get_base_action_url(self):
-        return reverse("bookmarks:archived.action")
-
-    def get_bookmark_query_set(self):
-        return queries.query_archived_bookmarks(
-            self.request.user, self.request.user_profile, self.search
-        )
+    request_context = ArchivedBookmarksContext
 
 
 class SharedBookmarkListContext(BookmarkListContext):
-    def get_base_url(self):
-        return reverse("bookmarks:shared")
-
-    def get_base_action_url(self):
-        return reverse("bookmarks:shared.action")
-
-    def get_bookmark_query_set(self):
-        user = User.objects.filter(username=self.search.user).first()
-        public_only = not self.request.user.is_authenticated
-        return queries.query_shared_bookmarks(
-            user, self.request.user_profile, self.search, public_only
-        )
+    request_context = SharedBookmarksContext
 
 
 class TagGroup:
@@ -191,7 +264,16 @@ class TagGroup:
         return f"<{self.char} TagGroup>"
 
     @staticmethod
-    def create_tag_groups(tags: Set[Tag]):
+    def create_tag_groups(mode: str, tags: Set[Tag]):
+        if mode == UserProfile.TAG_GROUPING_ALPHABETICAL:
+            return TagGroup._create_tag_groups_alphabetical(tags)
+        elif mode == UserProfile.TAG_GROUPING_DISABLED:
+            return TagGroup._create_tag_groups_disabled(tags)
+        else:
+            raise ValueError(f"{mode} is not a valid tag grouping mode")
+
+    @staticmethod
+    def _create_tag_groups_alphabetical(tags: Set[Tag]):
         # Ensure groups, as well as tags within groups, are ordered alphabetically
         sorted_tags = sorted(tags, key=lambda x: str.lower(x.name))
         group = None
@@ -216,9 +298,24 @@ class TagGroup:
             groups.append(cjk_group)
         return groups
 
+    @staticmethod
+    def _create_tag_groups_disabled(tags: Set[Tag]):
+        if len(tags) == 0:
+            return []
+
+        sorted_tags = sorted(tags, key=lambda x: str.lower(x.name))
+        group = TagGroup("Ungrouped")
+        for tag in sorted_tags:
+            group.tags.append(tag)
+
+        return [group]
+
 
 class TagCloudContext:
+    request_context = RequestContext
+
     def __init__(self, request: WSGIRequest) -> None:
+        request_context = self.request_context(request)
         user_profile = request.user_profile
 
         self.request = request
@@ -226,7 +323,7 @@ class TagCloudContext:
             self.request.GET, user_profile.search_preferences
         )
 
-        query_set = self.get_tag_query_set()
+        query_set = request_context.get_tag_query_set(self.search)
         tags = list(query_set)
         selected_tags = self.get_selected_tags(tags)
         unique_tags = utils.unique(tags, key=lambda x: str.lower(x.name))
@@ -235,15 +332,14 @@ class TagCloudContext:
         )
         has_selected_tags = len(unique_selected_tags) > 0
         unselected_tags = set(unique_tags).symmetric_difference(unique_selected_tags)
-        groups = TagGroup.create_tag_groups(unselected_tags)
+        groups = TagGroup.create_tag_groups(user_profile.tag_grouping, unselected_tags)
 
         self.tags = unique_tags
         self.groups = groups
         self.selected_tags = unique_selected_tags
         self.has_selected_tags = has_selected_tags
 
-    def get_tag_query_set(self):
-        raise Exception("Must be implemented by subclass")
+        self.refresh_url = request_context.tag_cloud_partial()
 
     def get_selected_tags(self, tags: List[Tag]):
         parsed_query = queries.parse_query_string(self.search.q)
@@ -256,26 +352,15 @@ class TagCloudContext:
 
 
 class ActiveTagCloudContext(TagCloudContext):
-    def get_tag_query_set(self):
-        return queries.query_bookmark_tags(
-            self.request.user, self.request.user_profile, self.search
-        )
+    request_context = ActiveBookmarksContext
 
 
 class ArchivedTagCloudContext(TagCloudContext):
-    def get_tag_query_set(self):
-        return queries.query_archived_bookmark_tags(
-            self.request.user, self.request.user_profile, self.search
-        )
+    request_context = ArchivedBookmarksContext
 
 
 class SharedTagCloudContext(TagCloudContext):
-    def get_tag_query_set(self):
-        user = User.objects.filter(username=self.search.user).first()
-        public_only = not self.request.user.is_authenticated
-        return queries.query_shared_bookmark_tags(
-            user, self.request.user_profile, self.search, public_only
-        )
+    request_context = SharedBookmarksContext
 
 
 class BookmarkAssetItem:
@@ -284,6 +369,7 @@ class BookmarkAssetItem:
 
         self.id = asset.id
         self.display_name = asset.display_name
+        self.asset_type = asset.asset_type
         self.content_type = asset.content_type
         self.file = asset.file
         self.file_size = asset.file_size
@@ -321,6 +407,7 @@ class BookmarkDetailsContext:
         self.profile = request.user_profile
         self.is_editable = bookmark.owner == user
         self.sharing_enabled = user_profile.enable_sharing
+        self.preview_image_enabled = user_profile.enable_preview_images
         self.show_link_icons = user_profile.enable_favicons and bookmark.favicon_file
         # For now hide files section if snapshots are not supported
         self.show_files = settings.LD_ENABLE_SNAPSHOTS
@@ -328,3 +415,15 @@ class BookmarkDetailsContext:
         self.assets = [
             BookmarkAssetItem(asset) for asset in bookmark.bookmarkasset_set.all()
         ]
+        self.has_pending_assets = any(
+            asset.status == BookmarkAsset.STATUS_PENDING for asset in self.assets
+        )
+        self.latest_snapshot = next(
+            (
+                asset
+                for asset in self.assets
+                if asset.asset.asset_type == BookmarkAsset.TYPE_SNAPSHOT
+                and asset.status == BookmarkAsset.STATUS_COMPLETE
+            ),
+            None,
+        )

bookmarks/views/settings.py

@@ -25,12 +25,10 @@ def general(request):
     profile_form = None
     enable_refresh_favicons = django_settings.LD_ENABLE_REFRESH_FAVICONS
     has_snapshot_support = django_settings.LD_ENABLE_SNAPSHOTS
-    update_profile_success_message = None
-    refresh_favicons_success_message = None
-    import_success_message = _find_message_with_tag(
+    success_message = _find_message_with_tag(
         messages.get_messages(request), "bookmark_import_success"
     )
-    import_errors_message = _find_message_with_tag(
+    error_message = _find_message_with_tag(
         messages.get_messages(request), "bookmark_import_errors"
     )
     version_info = get_version_info(get_ttl_hash())
@@ -38,12 +36,18 @@ def general(request):
     if request.method == "POST":
         if "update_profile" in request.POST:
             profile_form = update_profile(request)
-            update_profile_success_message = "Profile updated"
+            success_message = "Profile updated"
         if "refresh_favicons" in request.POST:
             tasks.schedule_refresh_favicons(request.user)
-            refresh_favicons_success_message = (
-                "Scheduled favicon update. This may take a while..."
-            )
+            success_message = "Scheduled favicon update. This may take a while..."
+        if "create_missing_html_snapshots" in request.POST:
+            count = tasks.create_missing_html_snapshots(request.user)
+            if count > 0:
+                success_message = (
+                    f"Queued {count} missing snapshots. This may take a while..."
+                )
+            else:
+                success_message = "No missing snapshots found."
 
     if not profile_form:
         profile_form = UserProfileForm(instance=request.user_profile)
@@ -55,10 +59,8 @@ def general(request):
             "form": profile_form,
             "enable_refresh_favicons": enable_refresh_favicons,
             "has_snapshot_support": has_snapshot_support,
-            "update_profile_success_message": update_profile_success_message,
-            "refresh_favicons_success_message": refresh_favicons_success_message,
-            "import_success_message": import_success_message,
-            "import_errors_message": import_errors_message,
+            "success_message": success_message,
+            "error_message": error_message,
             "version_info": version_info,
         },
     )
@@ -68,11 +70,16 @@ def update_profile(request):
     user = request.user
     profile = user.profile
     favicons_were_enabled = profile.enable_favicons
+    previews_were_enabled = profile.enable_preview_images
     form = UserProfileForm(request.POST, instance=profile)
     if form.is_valid():
         form.save()
+        # Load missing favicons if the feature was just enabled
         if profile.enable_favicons and not favicons_were_enabled:
             tasks.schedule_bookmarks_without_favicons(request.user)
+        # Load missing preview images if the feature was just enabled
+        if profile.enable_preview_images and not previews_were_enabled:
+            tasks.schedule_bookmarks_without_previews(request.user)
     return form
 
 

bootstrap.sh

@@ -7,6 +7,8 @@ LD_SERVER_PORT="${LD_SERVER_PORT:-9090}"
 mkdir -p data
 # Create favicon folder if it does not exist
 mkdir -p data/favicons
+# Create previews folder if it does not exist
+mkdir -p data/previews
 # Create assets folder if it does not exist
 mkdir -p data/assets
 
@@ -18,8 +20,10 @@ python manage.py migrate
 python manage.py enable_wal
 # Create initial superuser if defined in options / environment variables
 python manage.py create_initial_superuser
+# Migrate legacy background tasks to Huey
+python manage.py migrate_tasks
 
-# Ensure the DB folder is owned by the right user
+# Ensure folders are owned by the right user
 chown -R www-data: /etc/linkding/data
 
 # Start background task processor using supervisord, unless explicitly disabled

docker/alpine.Dockerfile

@@ -31,7 +31,8 @@ RUN pip install -U pip && pip install -r requirements.txt -r requirements.dev.tx
 COPY . .
 COPY --from=node-build /etc/linkding .
 # run Django part of the build
-RUN python manage.py compilescss && \
+RUN mkdir data && \
+    python manage.py compilescss && \
     python manage.py collectstatic --ignore=*.scss && \
     python manage.py compilescss --delete-files
 
@@ -98,8 +99,31 @@ CMD curl -f http://localhost:${LD_SERVER_PORT:-9090}/${LD_CONTEXT_PATH}health ||
 CMD ["./bootstrap.sh"]
 
 
+FROM node:18-alpine AS ublock-build
+WORKDIR /etc/linkding
+# Install necessary tools
+RUN apk add --no-cache curl jq unzip
+# Fetch the latest release tag
+# Download the library
+# Unzip the library
+RUN TAG=$(curl -sL https://api.github.com/repos/gorhill/uBlock/releases/latest | jq -r '.tag_name') && \
+    DOWNLOAD_URL=https://github.com/gorhill/uBlock/releases/download/$TAG/uBlock0_$TAG.chromium.zip && \
+    curl -L -o uBlock0.zip $DOWNLOAD_URL && \
+    unzip uBlock0.zip
+# Patch assets.json to enable easylist-cookies by default
+RUN curl -L -o ./uBlock0.chromium/assets/thirdparties/easylist/easylist-cookies.txt https://ublockorigin.github.io/uAssets/thirdparties/easylist-cookies.txt
+RUN jq '."assets.json" |= del(.cdnURLs) | ."assets.json".contentURL = ["assets/assets.json"] | ."fanboy-cookiemonster" |= del(.off) | ."fanboy-cookiemonster".contentURL += ["assets/thirdparties/easylist/easylist-cookies.txt"]' ./uBlock0.chromium/assets/assets.json > temp.json && \
+    mv temp.json ./uBlock0.chromium/assets/assets.json
+
+
 FROM linkding AS linkding-plus
-# install node, chromium and single-file
-RUN apk update && apk add nodejs npm chromium && npm install -g single-file-cli
+# install node, chromium
+RUN apk update && apk add nodejs npm chromium
+# install single-file from fork for now, which contains several hotfixes
+RUN npm install -g https://github.com/sissbruecker/single-file-cli/tarball/4c54b3bc704cfb3e96cec2d24854caca3df0b3b6
+# copy uBlock0
+COPY --from=ublock-build /etc/linkding/uBlock0.chromium uBlock0.chromium/
+# create chromium profile folder for user running background tasks
+RUN mkdir -p chromium-profile && chown -R www-data:www-data chromium-profile
 # enable snapshot support
 ENV LD_ENABLE_SNAPSHOTS=True

docker/default.Dockerfile

@@ -33,7 +33,8 @@ RUN pip install -U pip && pip install -r requirements.txt -r requirements.dev.tx
 COPY . .
 COPY --from=node-build /etc/linkding .
 # run Django part of the build
-RUN python manage.py compilescss && \
+RUN mkdir data && \
+    python manage.py compilescss && \
     python manage.py collectstatic --ignore=*.scss && \
     python manage.py compilescss --delete-files
 
@@ -95,8 +96,38 @@ CMD curl -f http://localhost:${LD_SERVER_PORT:-9090}/${LD_CONTEXT_PATH}health ||
 
 CMD ["./bootstrap.sh"]
 
+
+FROM node:18-alpine AS ublock-build
+WORKDIR /etc/linkding
+# Install necessary tools
+RUN apk add --no-cache curl jq unzip
+# Fetch the latest release tag
+# Download the library
+# Unzip the library
+RUN TAG=$(curl -sL https://api.github.com/repos/gorhill/uBlock/releases/latest | jq -r '.tag_name') && \
+    DOWNLOAD_URL=https://github.com/gorhill/uBlock/releases/download/$TAG/uBlock0_$TAG.chromium.zip && \
+    curl -L -o uBlock0.zip $DOWNLOAD_URL && \
+    unzip uBlock0.zip
+# Patch assets.json to enable easylist-cookies by default
+RUN curl -L -o ./uBlock0.chromium/assets/thirdparties/easylist/easylist-cookies.txt https://ublockorigin.github.io/uAssets/thirdparties/easylist-cookies.txt
+RUN jq '."assets.json" |= del(.cdnURLs) | ."assets.json".contentURL = ["assets/assets.json"] | ."fanboy-cookiemonster" |= del(.off) | ."fanboy-cookiemonster".contentURL += ["assets/thirdparties/easylist/easylist-cookies.txt"]' ./uBlock0.chromium/assets/assets.json > temp.json && \
+    mv temp.json ./uBlock0.chromium/assets/assets.json
+
+
 FROM linkding AS linkding-plus
-# install node, chromium and single-file
-RUN apt-get update && apt-get -y install nodejs npm chromium && npm install -g single-file-cli
+# install chromium
+RUN apt-get update && apt-get -y install chromium
+# install node
+ENV NODE_MAJOR=20
+RUN apt-get install -y gnupg2 apt-transport-https ca-certificates && \
+    curl -fsSL https://deb.nodesource.com/gpgkey/nodesource-repo.gpg.key | gpg --dearmor -o /usr/share/keyrings/nodesource.gpg && \
+    echo "deb [signed-by=/usr/share/keyrings/nodesource.gpg] https://deb.nodesource.com/node_$NODE_MAJOR.x nodistro main" | tee /etc/apt/sources.list.d/nodesource.list && \
+    apt-get update && apt-get install -y nodejs
+# install single-file from fork for now, which contains several hotfixes
+RUN npm install -g https://github.com/sissbruecker/single-file-cli/tarball/4c54b3bc704cfb3e96cec2d24854caca3df0b3b6
+# create chromium profile folder for user running background tasks
+RUN mkdir -p chromium-profile && chown -R www-data:www-data chromium-profile
+# copy uBlock0
+COPY --from=ublock-build /etc/linkding/uBlock0.chromium uBlock0.chromium/
 # enable snapshot support
 ENV LD_ENABLE_SNAPSHOTS=True

docs/Options.md

@@ -118,6 +118,46 @@ The following options can be configured:
 - `OIDC_RP_CLIENT_SECRET` - The client secret of the application.
 - `OIDC_RP_SIGN_ALGO` - The algorithm the OIDC provider uses to sign ID tokens. Default is `RS256`.
 - `OIDC_USE_PKCE` - Whether to use PKCE for the OIDC flow. Default is `True`.
+- `OIDC_VERIFY_SSL` - Whether to verify the SSL certificate of the OIDC provider. Set to `False` if using self-signed certificates or custom certificate authority. Default is `True`.
+
+<details>
+
+<summary>Authelia Example</summary>
+
+#### Linkding Configuration
+
+```bash
+LD_ENABLE_OIDC=True
+OIDC_OP_AUTHORIZATION_ENDPOINT=https://auth.example.com/api/oidc/authorization
+OIDC_OP_TOKEN_ENDPOINT=https://auth.example.com/api/oidc/token
+OIDC_OP_USER_ENDPOINT=https://auth.example.com/api/oidc/userinfo
+OIDC_OP_JWKS_ENDPOINT=https://auth.example.com/jwks.json
+OIDC_RP_CLIENT_ID=linkding
+OIDC_RP_CLIENT_SECRET=myClientSecret
+```
+#### Authelia Configuration
+
+```yaml
+identity_providers:
+  oidc:
+    # --- more OIDC provider configuration ---
+
+    clients:
+      - id: linkding
+        description: Linkding
+        # docker run --rm authelia/authelia:latest authelia crypto rand --length 64 --charset alphanumeric
+        secret: myClientSecret
+        public: false
+        token_endpoint_auth_method: client_secret_post
+        scopes:
+          - openid
+          - email
+          - profile
+        redirect_uris:
+          - https://linkding.example.com/oidc/callback/
+```
+
+</details>
 
 ### `LD_CSRF_TRUSTED_ORIGINS`
 
@@ -206,3 +246,20 @@ See the default URL for how to insert the placeholder to the favicon provider UR
 
 Alternative favicon providers:
 - DuckDuckGo: `https://icons.duckduckgo.com/ip3/{domain}.ico`
+
+
+### `LD_SINGLEFILE_TIMEOUT_SEC`
+
+Values: `Float` | Default =  60.0
+
+When creating HTML archive snapshots, control the timeout for how long to wait for the snapshot to complete, in `seconds`.
+Defaults to 60 seconds; on lower-powered hardware you may need to increase this value. 
+
+### `LD_SINGLEFILE_OPTIONS`
+
+Values: `String` | Default = None
+
+When creating HTML archive snapshots, pass additional options to the `single-file` application that is used to create snapshots.
+See `single-file --help` for complete list of arguments, or browse source: https://github.com/gildas-lormeau/single-file-cli/blob/master/options.js
+
+Example: `LD_SINGLEFILE_OPTIONS=--user-agent="Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:124.0) Gecko/20100101 Firefox/124.0"`

docs/backup.md

@@ -4,24 +4,57 @@ Linkding stores all data in the application's data folder.
 The full path to that folder in the Docker container is `/etc/linkding/data`.
 As described in the installation docs, you should mount the `/etc/linkding/data` folder to a folder on your host system.
 
-The data folder contains the following contents:
+The data folder contains the following contents that are relevant for backups:
 - `db.sqlite3` - the SQLite database
+- `assets` - folder that contains HTML snapshots of bookmarks
 - `favicons` - folder that contains downloaded favicons
+- `previews` - folder that contains downloaded preview images
 
 The following sections explain how to back up the individual contents.
 
-## Database
+## Full backup
 
-This section describes several methods on how to back up the contents of the SQLite database.
+linkding provides a CLI command to create a full backup of the data folder. This creates a zip file that contains backups of the database, assets, favicons, and preview images.
+
+> [!NOTE]
+> This method assumes that you are using the default SQLite database.
+> If you are using a different database, such as Postgres, you'll have to back up the database and other contents of the data folder manually.
+
+To create a full backup, execute the following command:
+```shell
+docker exec -it linkding python manage.py full_backup /etc/linkding/data/backup.zip
+```
+This creates a `backup.zip` file in the Docker container under `/etc/linkding/data`.
+
+To copy the backup file to your host system, execute the following command:
+```shell
+docker cp linkding:/etc/linkding/data/backup.zip backup.zip
+```
+This copies the backup file from the Docker container to the current folder on your host system.
+Now you can move that file to your backup location.
+
+To restore a backup: 
+- Extract the zip file in a folder of your new installation.
+- Rename the extracted folder to `data`.
+- When starting the Docker container, mount that folder to `/etc/linkding/data` as explained in the README.
+- Then start the Docker container.
+
+## Alternative backup methods
+
+If you can't use the full backup method, this section describes alternatives how to back up the individual contents of the data folder.
+
+### SQLite database backup
+
+linkding includes a CLI command for creating a backup copy of the database.
 
 > [!WARNING]
 > While the SQLite database is just a single file, it is not recommended to just copy that file.
 > This method is not transaction safe and may result in a [corrupted database](https://www.sqlite.org/howtocorrupt.html).
 > Use one of the backup methods described below.
 
-### Using the backup command
-
-linkding includes a CLI command for creating a backup copy of the database.
+> [!WARNING]
+> This method is deprecated and may be removed in the future.
+> Please use the full backup method described above.
 
 To create a backup, execute the following command:
 ```shell
@@ -38,12 +71,12 @@ Now you can move that file to your backup location.
 
 To restore the backup, just copy the backup file to the data folder of your new installation and rename it to `db.sqlite3`. Then start the Docker container.
 
-### Using the SQLite dump function
+### SQLite database SQL dump
 
 Requires [SQLite](https://www.sqlite.org/index.html) to be installed on your host system.
 
 With this method you create a plain text file with the SQL statements to recreate the SQLite database.
-To create a backup, execute the following command in the data folder:
+To create a backup, execute the following command in the data folder on your host system:
 ```shell
 sqlite3 db.sqlite3 .dump > backup.sql
 ```
@@ -56,9 +89,9 @@ Using git, you can commit the changes, followed by a git push to a remote reposi
 This is the least technical option to back up bookmarks, but has several limitations:
 - It does not export user profiles.
 - It only exports your own bookmarks, not those of other users.
-- It does not export archived bookmarks.
 - It does not export URLs of snapshots on the Internet Archive Wayback machine.
-- It does not export favicons.
+- It does not export HTML snapshots of bookmarks. Even if you backup and restore the assets folder, the bookmarks will not be linked to the snapshots anymore.
+- It does not export favicons or preview images.
 
 Only use this method if you are fine with the above limitations.
 
@@ -70,7 +103,16 @@ To restore bookmarks, open the general settings on your new installation.
 In the Import section, click on the *Choose file* button to select the HTML file you downloaded before.
 Then click on the *Import* button to import the bookmarks.
 
-## Favicons
+### Assets
+
+If you are using the HTML snapshots feature, you should also do backups of the `assets` folder.
+It contains the HTML snapshots files of your bookmarks which are referenced from the database.
+
+To back up the assets, then you have to copy the `assets` folder to your backup location.
+
+To restore the assets, copy the `assets` folder back to the data folder of your new installation.
+
+### Favicons
 
 Doing a backup of the icons is optional, as they can be downloaded again.
 

package-lock.json

@@ -1,12 +1,12 @@
 {
   "name": "linkding",
-  "version": "1.25.0",
+  "version": "1.30.0",
   "lockfileVersion": 3,
   "requires": true,
   "packages": {
     "": {
       "name": "linkding",
-      "version": "1.25.0",
+      "version": "1.30.0",
       "license": "MIT",
       "dependencies": {
         "@rollup/plugin-node-resolve": "^15.2.3",

package.json

@@ -1,6 +1,6 @@
 {
   "name": "linkding",
-  "version": "1.27.0",
+  "version": "1.31.0",
   "description": "",
   "main": "index.js",
   "scripts": {
@@ -28,5 +28,6 @@
   },
   "devDependencies": {
     "prettier": "^3.0.2"
-  }
+  },
+  "web-types": "./web-types.json"
 }

requirements.dev.txt

@@ -54,7 +54,7 @@ rcssmin==1.1.1
     # via django-compressor
 rjsmin==1.2.1
     # via django-compressor
-sqlparse==0.4.4
+sqlparse==0.5.0
     # via
     #   django
     #   django-debug-toolbar

requirements.in

@@ -5,8 +5,8 @@ Django
 django-registration
 django-sass-processor
 django-widget-tweaks
-django4-background-tasks
 djangorestframework
+huey
 Markdown
 mozilla-django-oidc
 psycopg2-binary

requirements.txt

@@ -39,11 +39,11 @@ django-sass-processor==1.4
     # via -r requirements.in
 django-widget-tweaks==1.5.0
     # via -r requirements.in
-django4-background-tasks==1.2.9
-    # via -r requirements.in
 djangorestframework==3.14.0
     # via -r requirements.in
-idna==3.6
+huey==2.5.0
+    # via -r requirements.in
+idna==3.7
     # via requests
 josepy==1.14.0
     # via mozilla-django-oidc
@@ -61,7 +61,7 @@ python-dateutil==2.8.2
     # via -r requirements.in
 pytz==2023.3.post1
     # via djangorestframework
-requests==2.31.0
+requests==2.32.0
     # via
     #   -r requirements.in
     #   mozilla-django-oidc
@@ -69,11 +69,10 @@ requests==2.31.0
 six==1.16.0
     # via
     #   bleach
-    #   django4-background-tasks
     #   python-dateutil
 soupsieve==2.5
     # via beautifulsoup4
-sqlparse==0.4.4
+sqlparse==0.5.0
     # via django
 supervisor==4.2.5
     # via -r requirements.in