Bump version

* allow configuring landing page

* add tests

.dockerignore

@@ -5,7 +5,6 @@
 !/bookmarks
 !/siteroot
 
-!/background-tasks-wrapper.sh
 !/bootstrap.sh
 !/LICENSE.txt
 !/manage.py

.github/workflows/main.yaml

@@ -24,7 +24,9 @@ jobs:
       - name: Install Node dependencies
         run: npm ci
       - name: Setup Python environment
-        run: pip install -r requirements.txt -r requirements.dev.txt
+        run: |
+          pip install -r requirements.txt -r requirements.dev.txt
+          mkdir data
       - name: Run tests
         run: python manage.py test bookmarks.tests
   e2e_tests:
@@ -47,6 +49,7 @@ jobs:
         run: |
           pip install -r requirements.txt -r requirements.dev.txt
           playwright install chromium
+          mkdir data
       - name: Run build
         run: |
           npm run build

.gitignore

@@ -191,3 +191,8 @@ typings/
 /tmp
 # Database file
 /data
+# ublock + chromium
+/uBlock0.chromium
+/chromium-profile
+# direnv
+/.direnv

CHANGELOG.md

@@ -1,5 +1,140 @@
 # Changelog
 
+## v1.31.1 (30/08/2024)
+
+### What's Changed
+* Include favicons and thumbnails in REST API by @sissbruecker in https://github.com/sissbruecker/linkding/pull/763
+* Add Pinkt to the Community section by @fibelatti in https://github.com/sissbruecker/linkding/pull/772
+* removed version line from docker compose yaml by @volumedata21 in https://github.com/sissbruecker/linkding/pull/800
+* Add resource linkding logo by @QYG2297248353 in https://github.com/sissbruecker/linkding/pull/788
+* Allow use of standard docker `TZ` env var by @watsonbox in https://github.com/sissbruecker/linkding/pull/765
+* Add OCI source annotation to link back to source repo by @Ramblurr in https://github.com/sissbruecker/linkding/pull/701
+* Generate fallback URLs for web archive links by @sissbruecker in https://github.com/sissbruecker/linkding/pull/804
+* Fix overflow in settings page by @sissbruecker in https://github.com/sissbruecker/linkding/pull/805
+* Bump django from 5.0.3 to 5.0.8 by @dependabot in https://github.com/sissbruecker/linkding/pull/795
+* Bump certifi from 2023.11.17 to 2024.7.4 by @dependabot in https://github.com/sissbruecker/linkding/pull/775
+* Bump djangorestframework from 3.14.0 to 3.15.2 by @dependabot in https://github.com/sissbruecker/linkding/pull/769
+* Bump urllib3 from 2.1.0 to 2.2.2 by @dependabot in https://github.com/sissbruecker/linkding/pull/762
+
+### New Contributors
+* @fibelatti made their first contribution in https://github.com/sissbruecker/linkding/pull/772
+* @volumedata21 made their first contribution in https://github.com/sissbruecker/linkding/pull/800
+* @QYG2297248353 made their first contribution in https://github.com/sissbruecker/linkding/pull/788
+* @watsonbox made their first contribution in https://github.com/sissbruecker/linkding/pull/765
+* @Ramblurr made their first contribution in https://github.com/sissbruecker/linkding/pull/701
+
+**Full Changelog**: https://github.com/sissbruecker/linkding/compare/v1.31.0...v1.31.1
+
+---
+
+## v1.31.0 (16/06/2024)
+
+### What's Changed
+* Add support for bookmark thumbnails by @vslinko in https://github.com/sissbruecker/linkding/pull/721
+* Automatically add tags to bookmarks based on URL pattern by @vslinko in https://github.com/sissbruecker/linkding/pull/736
+* Load bookmark thumbnails after import by @vslinko in https://github.com/sissbruecker/linkding/pull/724
+* Load missing thumbnails after enabling the feature by @sissbruecker in https://github.com/sissbruecker/linkding/pull/725
+* Thumbnails lazy loading by @vslinko in https://github.com/sissbruecker/linkding/pull/734
+* Add option for disabling tag grouping by @vslinko in https://github.com/sissbruecker/linkding/pull/735
+* Preview auto tags in bookmark form by @sissbruecker in https://github.com/sissbruecker/linkding/pull/737
+* Hide tooltip on mobile by @vslinko in https://github.com/sissbruecker/linkding/pull/733
+* Bump requests from 2.31.0 to 2.32.0 by @dependabot in https://github.com/sissbruecker/linkding/pull/740
+
+### New Contributors
+* @vslinko made their first contribution in https://github.com/sissbruecker/linkding/pull/721
+
+**Full Changelog**: https://github.com/sissbruecker/linkding/compare/v1.30.0...v1.31.0
+
+---
+
+## v1.30.0 (20/04/2024)
+
+### What's Changed
+* Add reader mode by @sissbruecker in https://github.com/sissbruecker/linkding/pull/703
+* Allow uploading custom files for bookmarks by @sissbruecker in https://github.com/sissbruecker/linkding/pull/713
+* Add option for marking bookmarks as unread by default by @ab623 in https://github.com/sissbruecker/linkding/pull/706
+* Make blocking cookie banners more reliable by @sissbruecker in https://github.com/sissbruecker/linkding/pull/699
+* Close bookmark details with escape by @sissbruecker in https://github.com/sissbruecker/linkding/pull/702
+* Show proper name for bookmark assets in admin by @ab623 in https://github.com/sissbruecker/linkding/pull/708
+* Bump sqlparse from 0.4.4 to 0.5.0 by @dependabot in https://github.com/sissbruecker/linkding/pull/704
+
+### New Contributors
+* @ab623 made their first contribution in https://github.com/sissbruecker/linkding/pull/706
+
+**Full Changelog**: https://github.com/sissbruecker/linkding/compare/v1.29.0...v1.30.0
+
+---
+
+## v1.29.0 (14/04/2024)
+
+### What's Changed
+* Remove ads and cookie banners from HTML snapshots by @sissbruecker in https://github.com/sissbruecker/linkding/pull/695
+* Add button for creating missing HTML snapshots by @sissbruecker in https://github.com/sissbruecker/linkding/pull/696
+* Refresh file list when there are queued snapshots by @sissbruecker in https://github.com/sissbruecker/linkding/pull/697
+* Bump idna from 3.6 to 3.7 by @dependabot in https://github.com/sissbruecker/linkding/pull/694
+
+
+**Full Changelog**: https://github.com/sissbruecker/linkding/compare/v1.28.0...v1.29.0
+
+---
+
+## v1.28.0 (09/04/2024)
+
+### What's Changed
+* Add option to disable SSL verification for OIDC by @akaSyntaax in https://github.com/sissbruecker/linkding/pull/684
+* Add full backup method by @sissbruecker in https://github.com/sissbruecker/linkding/pull/686
+* Truncate snapshot filename for long URLs by @sissbruecker in https://github.com/sissbruecker/linkding/pull/687
+* Add option for customizing single-file timeout by @pettijohn in https://github.com/sissbruecker/linkding/pull/688
+* Add option for passing arguments to single-file command by @pettijohn in https://github.com/sissbruecker/linkding/pull/691
+* Fix typo by @tianheg in https://github.com/sissbruecker/linkding/pull/689
+
+### New Contributors
+* @akaSyntaax made their first contribution in https://github.com/sissbruecker/linkding/pull/684
+* @pettijohn made their first contribution in https://github.com/sissbruecker/linkding/pull/688
+
+**Full Changelog**: https://github.com/sissbruecker/linkding/compare/v1.27.1...v1.28.0
+
+---
+
+## v1.27.1 (07/04/2024)
+
+### What's Changed
+* Fix HTML snapshot errors related to single-file-cli by @sissbruecker in https://github.com/sissbruecker/linkding/pull/683
+* Replace django-background-tasks with huey by @sissbruecker in https://github.com/sissbruecker/linkding/pull/657
+* Add Authelia OIDC example to docs by @hugo-vrijswijk in https://github.com/sissbruecker/linkding/pull/675
+
+
+**Full Changelog**: https://github.com/sissbruecker/linkding/compare/v1.27.0...v1.27.1
+
+---
+
+## v1.27.0 (01/04/2024)
+
+### What's Changed
+* Archive snapshots of websites locally by @sissbruecker in https://github.com/sissbruecker/linkding/pull/672
+* Add Railway hosting option by @tianheg in https://github.com/sissbruecker/linkding/pull/661
+* Add how to for increasing the font size by @sissbruecker in https://github.com/sissbruecker/linkding/pull/667
+
+### New Contributors
+* @tianheg made their first contribution in https://github.com/sissbruecker/linkding/pull/661
+
+**Full Changelog**: https://github.com/sissbruecker/linkding/compare/v1.26.0...v1.27.0
+
+---
+
+## v1.26.0 (30/03/2024)
+
+### What's Changed
+* Add option for showing bookmark description as separate block by @sissbruecker in https://github.com/sissbruecker/linkding/pull/663
+* Add bookmark details view by @sissbruecker in https://github.com/sissbruecker/linkding/pull/665
+* Make bookmark list actions configurable by @sissbruecker in https://github.com/sissbruecker/linkding/pull/666
+* Bump black from 24.1.1 to 24.3.0 by @dependabot in https://github.com/sissbruecker/linkding/pull/662
+
+
+**Full Changelog**: https://github.com/sissbruecker/linkding/compare/v1.25.0...v1.26.0
+
+---
+
 ## v1.25.0 (18/03/2024)
 
 ### What's Changed

Makefile

@@ -7,7 +7,7 @@ tasks:
 	python manage.py process_tasks
 
 test:
-	pytest
+	pytest -n auto
 
 format:
 	black bookmarks

README.md

@@ -33,22 +33,19 @@ The name comes from:
 **Feature Overview:**
 - Clean UI optimized for readability
 - Organize bookmarks with tags
-- Add notes using Markdown
+- Bulk editing, Markdown notes, read it later functionality
-- Read it later functionality
+- Share bookmarks with other users or guests
-- Share bookmarks with other users
-- Bulk editing
 - Automatically provides titles, descriptions and icons of bookmarked websites
-- Automatically creates snapshots of bookmarked websites on [the Internet Archive Wayback Machine](https://archive.org/web/)
+- Automatically archive websites, either as local HTML file or on Internet Archive
 - Import and export bookmarks in Netscape HTML format
 - Installable as a Progressive Web App (PWA)
 - Extensions for [Firefox](https://addons.mozilla.org/firefox/addon/linkding-extension/) and [Chrome](https://chrome.google.com/webstore/detail/linkding-extension/beakmhbijpdhipnjhnclmhgjlddhidpe), as well as a bookmarklet
-- Light and dark themes
 - SSO support via OIDC or authentication proxies
 - REST API for developing 3rd party apps
 - Admin panel for user self-service and raw data access
 
 
-**Demo:** https://demo.linkding.link/ ([see here](https://github.com/sissbruecker/linkding/issues/408) if you have trouble accessing it)
+**Demo:** https://demo.linkding.link/
 
 **Screenshot:**
 
@@ -62,27 +59,45 @@ The Docker image is compatible with ARM platforms, so it can be run on a Raspber
 linkding uses an SQLite database by default.
 Alternatively linkding supports PostgreSQL, see the [database options](docs/Options.md#LD_DB_ENGINE) for more information.
 
-<details>
-
-<summary>🧪 Alpine-based image</summary>
-
-The default Docker image (`latest` tag) is based on a slim variant of Debian Linux.
-Alternatively, there is an image based on Alpine Linux (`latest-alpine` tag) which has a smaller size, resulting in a smaller download and less disk space required.
-The Alpine image is currently about 45 MB in compressed size, compared to about 130 MB for the Debian image.
-
-To use it, replace the `latest` tag with `latest-alpine`, either in the CLI command below when using Docker, or in the `docker-compose.yml` file when using docker-compose.
-
-> [!WARNING]
-> The image is currently considered experimental in order to gather feedback and iron out any issues.
-> Only use it if you are comfortable running experimental software or want to help out with testing.
-> While there should be no issues with creating new installations, there might be issues when migrating existing installations.
-> If you plan to migrate your existing installation, make sure to create proper [backups](https://github.com/sissbruecker/linkding/blob/master/docs/backup.md) first.
-
-</details>
-
 ###  Using Docker
 
-To install linkding using Docker you can just run the [latest image](https://hub.docker.com/repository/docker/sissbruecker/linkding) from Docker Hub:
+The Docker image comes in several variants. To use a different image than the default, replace `latest` with the desired tag in the commands below, or in the docker-compose file.
+
+<table>
+  <thead>
+    <tr>
+      <th>Tag</th>
+      <th>Description</th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr>
+      <td><code>latest</code></td>
+      <td>Provides the basic functionality of linkding</td>
+    </tr>
+    <tr>
+      <td><code>latest-plus</code></td>
+      <td>
+        Includes feature for archiving websites as HTML snapshots
+        <ul>
+          <li>Significantly larger image size as it includes a Chromium installation</li>
+          <li>Requires more runtime memory to run Chromium</li>
+          <li>Requires more disk space for storing HTML snapshots</li>
+        </ul>            
+      </td>
+    </tr>
+    <tr>
+      <td><code>latest-alpine</code></td>
+      <td><code>latest</code>, but based on Alpine Linux. 🧪 Experimental</td>
+    </tr>    
+    <tr>
+      <td><code>latest-plus-alpine</code></td>
+      <td><code>latest-plus</code>, but based on Alpine Linux. 🧪 Experimental</td>
+    </tr>    
+  </tbody>
+</table>
+
+To install linkding using Docker you can just run the image from [Docker Hub](https://hub.docker.com/repository/docker/sissbruecker/linkding):
 ```shell
 docker run --name linkding -p 9090:9090 -v {host-data-folder}:/etc/linkding/data -d sissbruecker/linkding:latest
 ```
@@ -184,6 +199,7 @@ Self-hosting web applications still requires a lot of technical know-how and com
 - [linkding on fly.io](https://github.com/fspoettel/linkding-on-fly) - Guide for hosting a linkding installation on [fly.io](https://fly.io). By [fspoettel](https://github.com/fspoettel)
 - [PikaPods.com](https://www.pikapods.com/) - Managed hosting for linkding, EU and US regions available. [1-click setup link](https://www.pikapods.com/pods?run=linkding) ([Disclosure](#pikapods))
 - [CapRover](https://caprover.com/) - Linkding is included as a default one-click app
+- [linkding on railway.app](https://github.com/tianheg/linkding-on-railway) - Guide for hosting a linkding installation on [railway.app](https://railway.app/). By [tianheg](https://github.com/tianheg)
 
 ## Documentation
 
@@ -221,6 +237,7 @@ This section lists community projects around using linkding, in alphabetical ord
 - [Linkdy](https://github.com/JGeek00/linkdy): An open source mobile and desktop (not yet) client created with Flutter. Available at the [Google Play Store](https://play.google.com/store/apps/details?id=com.jgeek00.linkdy). By [JGeek00](https://github.com/JGeek00).
 - [LinkThing](https://apps.apple.com/us/app/linkthing/id1666031776) An iOS client for linkding. By [amoscardino](https://github.com/amoscardino)
 - [Open all links bookmarklet](https://gist.github.com/ukcuddlyguy/336dd7339e6d35fc64a75ccfc9323c66) A browser bookmarklet to open all links on the current Linkding page in new tabs. By [ukcuddlyguy](https://github.com/ukcuddlyguy)
+- [Pinkt](https://github.com/fibelatti/pinboard-kotlin) An Android client for linkding. By [fibelatti](https://github.com/fibelatti)
 - [Postman collection](https://gist.github.com/gingerbeardman/f0b42502f3bc9344e92ce63afd4360d3) a group of saved request templates for API testing. By [gingerbeardman](https://github.com/gingerbeardman)
 
 ## Acknowledgements + Donations

assets/logo.svg

@@ -0,0 +1 @@
+<svg clip-rule="evenodd" fill-rule="evenodd" height="512" stroke-linejoin="round" stroke-miterlimit="1.5" viewBox="0 0 512 512" width="512" xmlns="http://www.w3.org/2000/svg"><circle cx="255.0164" cy="254.9236" fill="#5856e0" r="224.78528" stroke-width="1.18"/><g fill="none" stroke="#fff" stroke-width="31.25"><path d="m1244.39 1293.95v199.64s-.81 67.89 74.9 68.88c75.98.99 74.88-68.88 74.88-68.88v-199.64" transform="matrix(.70710678 .70710678 -.70710678 .70710678 284.139117 -1684.198509)"/><path d="m1244.39 1293.95v199.64s-.81 67.89 74.9 68.88c75.98.99 74.88-68.88 74.88-68.88v-199.64" transform="matrix(-.70957074 -.70463421 .70463421 -.70957074 235.113139 2195.434643)"/></g></svg>

background-tasks-wrapper.sh

@@ -1,5 +0,0 @@
-#!/usr/bin/env bash
-# Wrapper script used by supervisord to first clear task locks before starting the background task processor
-
-python manage.py clean_tasks
-exec python manage.py process_tasks

bookmarks/admin.py

@@ -1,22 +1,96 @@
-from background_task.admin import TaskAdmin, CompletedTaskAdmin
-from background_task.models import Task, CompletedTask
 from django.contrib import admin, messages
 from django.contrib.admin import AdminSite
 from django.contrib.auth.admin import UserAdmin
 from django.contrib.auth.models import User
+from django.core.paginator import Paginator
 from django.db.models import Count, QuerySet
+from django.shortcuts import render
+from django.urls import path
 from django.utils.translation import ngettext, gettext
+from huey.contrib.djhuey import HUEY as huey
 from rest_framework.authtoken.admin import TokenAdmin
 from rest_framework.authtoken.models import TokenProxy
 
-from bookmarks.models import Bookmark, Tag, UserProfile, Toast, FeedToken
+from bookmarks.models import Bookmark, BookmarkAsset, Tag, UserProfile, Toast, FeedToken
 from bookmarks.services.bookmarks import archive_bookmark, unarchive_bookmark
 
 
+# Custom paginator to paginate through Huey tasks
+class TaskPaginator(Paginator):
+    def __init__(self):
+        super().__init__(self, 100)
+        self.task_count = huey.storage.queue_size()
+
+    @property
+    def count(self):
+        return self.task_count
+
+    def page(self, number):
+        limit = self.per_page
+        offset = (number - 1) * self.per_page
+        return self._get_page(
+            self.enqueued_items(limit, offset),
+            number,
+            self,
+        )
+
+    # Copied from Huey's SqliteStorage with some modifications to allow pagination
+    def enqueued_items(self, limit, offset):
+        to_bytes = lambda b: bytes(b) if not isinstance(b, bytes) else b
+        sql = "select data from task where queue=? order by priority desc, id limit ? offset ?"
+        params = (huey.storage.name, limit, offset)
+
+        serialized_tasks = [
+            to_bytes(i) for i, in huey.storage.sql(sql, params, results=True)
+        ]
+        return [huey.deserialize_task(task) for task in serialized_tasks]
+
+
+# Custom view to display Huey tasks in the admin
+def background_task_view(request):
+    page_number = int(request.GET.get("p", 1))
+    paginator = TaskPaginator()
+    page = paginator.get_page(page_number)
+    page_range = paginator.get_elided_page_range(page_number, on_each_side=2, on_ends=2)
+    context = {
+        **linkding_admin_site.each_context(request),
+        "title": "Background tasks",
+        "page": page,
+        "page_range": page_range,
+        "tasks": page.object_list,
+    }
+    return render(request, "admin/background_tasks.html", context)
+
+
 class LinkdingAdminSite(AdminSite):
     site_header = "linkding administration"
     site_title = "linkding Admin"
 
+    def get_urls(self):
+        urls = super().get_urls()
+        custom_urls = [
+            path("tasks/", background_task_view, name="background_tasks"),
+        ]
+        return custom_urls + urls
+
+    def get_app_list(self, request, app_label=None):
+        app_list = super().get_app_list(request, app_label)
+        app_list += [
+            {
+                "name": "Huey",
+                "app_label": "huey_app",
+                "models": [
+                    {
+                        "name": "Queued tasks",
+                        "object_name": "background_tasks",
+                        "admin_url": "/admin/tasks/",
+                        "view_only": True,
+                    }
+                ],
+            }
+        ]
+        return app_list
+
 
 class AdminBookmark(admin.ModelAdmin):
     list_display = ("resolved_title", "url", "is_archived", "owner", "date_added")
@@ -125,6 +199,19 @@ class AdminBookmark(admin.ModelAdmin):
         )
 
 
+class AdminBookmarkAsset(admin.ModelAdmin):
+    @admin.display(description="Display Name")
+    def custom_display_name(self, obj):
+        return str(obj)
+
+    list_display = ("custom_display_name", "date_created", "status")
+    search_fields = (
+        "custom_display_name",
+        "file",
+    )
+    list_filter = ("status",)
+
+
 class AdminTag(admin.ModelAdmin):
     list_display = ("name", "bookmarks_count", "owner", "date_added")
     search_fields = ("name", "owner__username")
@@ -200,10 +287,9 @@ class AdminFeedToken(admin.ModelAdmin):
 
 linkding_admin_site = LinkdingAdminSite()
 linkding_admin_site.register(Bookmark, AdminBookmark)
+linkding_admin_site.register(BookmarkAsset, AdminBookmarkAsset)
 linkding_admin_site.register(Tag, AdminTag)
 linkding_admin_site.register(User, AdminCustomUser)
 linkding_admin_site.register(TokenProxy, TokenAdmin)
 linkding_admin_site.register(Toast, AdminToast)
 linkding_admin_site.register(FeedToken, AdminFeedToken)
-linkding_admin_site.register(Task, TaskAdmin)
-linkding_admin_site.register(CompletedTask, CompletedTaskAdmin)

bookmarks/api/routes.py

@@ -1,3 +1,5 @@
+import logging
+
 from rest_framework import viewsets, mixins, status
 from rest_framework.decorators import action
 from rest_framework.permissions import AllowAny
@@ -11,6 +13,7 @@ from bookmarks.api.serializers import (
     UserProfileSerializer,
 )
 from bookmarks.models import Bookmark, BookmarkSearch, Tag, User
+from bookmarks.services import auto_tagging
 from bookmarks.services.bookmarks import (
     archive_bookmark,
     unarchive_bookmark,
@@ -18,6 +21,8 @@ from bookmarks.services.bookmarks import (
 )
 from bookmarks.services.website_loader import WebsiteMetadata
 
+logger = logging.getLogger(__name__)
+
 
 class BookmarkViewSet(
     viewsets.GenericViewSet,
@@ -51,7 +56,7 @@ class BookmarkViewSet(
         return Bookmark.objects.all().filter(owner=user)
 
     def get_serializer_context(self):
-        return {"user": self.request.user}
+        return {"request": self.request, "user": self.request.user}
 
     @action(methods=["get"], detail=False)
     def archived(self, request):
@@ -59,8 +64,8 @@ class BookmarkViewSet(
         search = BookmarkSearch.from_request(request.GET)
         query_set = queries.query_archived_bookmarks(user, user.profile, search)
         page = self.paginate_queryset(query_set)
-        serializer = self.get_serializer_class()
+        serializer = self.get_serializer(page, many=True)
-        data = serializer(page, many=True).data
+        data = serializer.data
         return self.get_paginated_response(data)
 
     @action(methods=["get"], detail=False)
@@ -72,8 +77,8 @@ class BookmarkViewSet(
             user, request.user_profile, search, public_only
         )
         page = self.paginate_queryset(query_set)
-        serializer = self.get_serializer_class()
+        serializer = self.get_serializer(page, many=True)
-        data = serializer(page, many=True).data
+        data = serializer.data
         return self.get_paginated_response(data)
 
     @action(methods=["post"], detail=True)
@@ -99,13 +104,32 @@ class BookmarkViewSet(
         # Either return metadata from existing bookmark, or scrape from URL
         if bookmark:
             metadata = WebsiteMetadata(
-                url, bookmark.website_title, bookmark.website_description
+                url,
+                bookmark.website_title,
+                bookmark.website_description,
+                None,
             )
         else:
             metadata = website_loader.load_website_metadata(url)
 
+        # Return tags that would be automatically applied to the bookmark
+        profile = request.user.profile
+        auto_tags = []
+        if profile.auto_tagging_rules:
+            try:
+                auto_tags = auto_tagging.get_tags(profile.auto_tagging_rules, url)
+            except Exception as e:
+                logger.error(
+                    f"Failed to auto-tag bookmark. url={bookmark.url}",
+                    exc_info=e,
+                )
+
         return Response(
-            {"bookmark": existing_bookmark_data, "metadata": metadata.to_dict()},
+            {
+                "bookmark": existing_bookmark_data,
+                "metadata": metadata.to_dict(),
+                "auto_tags": auto_tags,
+            },
             status=status.HTTP_200_OK,
         )
 

bookmarks/api/serializers.py

@@ -1,4 +1,5 @@
 from django.db.models import prefetch_related_objects
+from django.templatetags.static import static
 from rest_framework import serializers
 from rest_framework.serializers import ListSerializer
 
@@ -31,6 +32,8 @@ class BookmarkSerializer(serializers.ModelSerializer):
             "website_title",
             "website_description",
             "web_archive_snapshot_url",
+            "favicon_url",
+            "preview_image_url",
             "is_archived",
             "unread",
             "shared",
@@ -42,6 +45,8 @@ class BookmarkSerializer(serializers.ModelSerializer):
             "website_title",
             "website_description",
             "web_archive_snapshot_url",
+            "favicon_url",
+            "preview_image_url",
             "date_added",
             "date_modified",
         ]
@@ -56,6 +61,24 @@ class BookmarkSerializer(serializers.ModelSerializer):
     shared = serializers.BooleanField(required=False, default=False)
     # Override readonly tag_names property to allow passing a list of tag names to create/update
     tag_names = TagListField(required=False, default=[])
+    favicon_url = serializers.SerializerMethodField()
+    preview_image_url = serializers.SerializerMethodField()
+
+    def get_favicon_url(self, obj: Bookmark):
+        if not obj.favicon_file:
+            return None
+        request = self.context.get("request")
+        favicon_file_path = static(obj.favicon_file)
+        favicon_url = request.build_absolute_uri(favicon_file_path)
+        return favicon_url
+
+    def get_preview_image_url(self, obj: Bookmark):
+        if not obj.preview_image_file:
+            return None
+        request = self.context.get("request")
+        preview_image_file_path = static(obj.preview_image_file)
+        preview_image_url = request.build_absolute_uri(preview_image_file_path)
+        return preview_image_url
 
     def create(self, validated_data):
         bookmark = Bookmark()

bookmarks/context_processors.py

@@ -18,19 +18,5 @@ def toasts(request):
     }
 
 
-def public_shares(request):
-    # Only check for public shares for anonymous users
-    if not request.user.is_authenticated:
-        query_set = queries.query_shared_bookmarks(
-            None, request.user_profile, BookmarkSearch(), True
-        )
-        has_public_shares = query_set.count() > 0
-        return {
-            "has_public_shares": has_public_shares,
-        }
-
-    return {}
-
-
 def app_version(request):
     return {"app_version": utils.app_version}

bookmarks/e2e/e2e_test_bookmark_details_modal.py

@@ -1,3 +1,4 @@
+from django.test import override_settings
 from django.urls import reverse
 from playwright.sync_api import sync_playwright, expect
 
@@ -33,6 +34,11 @@ class BookmarkDetailsModalE2ETestCase(LinkdingE2ETestCase):
             overlay.click(position={"x": 0, "y": 0})
             expect(details_modal).to_be_hidden()
 
+            # close with escape
+            details_modal = self.open_details_modal(bookmark)
+            self.page.keyboard.press("Escape")
+            expect(details_modal).to_be_hidden()
+
     def test_toggle_archived(self):
         bookmark = self.setup_bookmark()
 
@@ -44,14 +50,17 @@ class BookmarkDetailsModalE2ETestCase(LinkdingE2ETestCase):
             details_modal = self.open_details_modal(bookmark)
             details_modal.get_by_text("Archived", exact=False).click()
             expect(self.locate_bookmark(bookmark.title)).not_to_be_visible()
+            self.assertReloads(0)
 
             # unarchive
             url = reverse("bookmarks:archived")
             self.page.goto(self.live_server_url + url)
+            self.resetReloads()
 
             details_modal = self.open_details_modal(bookmark)
             details_modal.get_by_text("Archived", exact=False).click()
             expect(self.locate_bookmark(bookmark.title)).not_to_be_visible()
+            self.assertReloads(0)
 
     def test_toggle_unread(self):
         bookmark = self.setup_bookmark()
@@ -66,11 +75,13 @@ class BookmarkDetailsModalE2ETestCase(LinkdingE2ETestCase):
             details_modal.get_by_text("Unread").click()
             bookmark_item = self.locate_bookmark(bookmark.title)
             expect(bookmark_item.get_by_text("Unread")).to_be_visible()
+            self.assertReloads(0)
 
             # mark as read
             details_modal.get_by_text("Unread").click()
             bookmark_item = self.locate_bookmark(bookmark.title)
             expect(bookmark_item.get_by_text("Unread")).not_to_be_visible()
+            self.assertReloads(0)
 
     def test_toggle_shared(self):
         profile = self.get_or_create_test_user().profile
@@ -89,11 +100,13 @@ class BookmarkDetailsModalE2ETestCase(LinkdingE2ETestCase):
             details_modal.get_by_text("Shared").click()
             bookmark_item = self.locate_bookmark(bookmark.title)
             expect(bookmark_item.get_by_text("Shared")).to_be_visible()
+            self.assertReloads(0)
 
             # unshare bookmark
             details_modal.get_by_text("Shared").click()
             bookmark_item = self.locate_bookmark(bookmark.title)
             expect(bookmark_item.get_by_text("Shared")).not_to_be_visible()
+            self.assertReloads(0)
 
     def test_edit_return_url(self):
         bookmark = self.setup_bookmark()
@@ -131,3 +144,33 @@ class BookmarkDetailsModalE2ETestCase(LinkdingE2ETestCase):
             expect(self.locate_bookmark(bookmark.title)).not_to_be_visible()
 
         self.assertEqual(Bookmark.objects.count(), 0)
+
+    @override_settings(LD_ENABLE_SNAPSHOTS=True)
+    def test_create_snapshot_remove_snapshot(self):
+        bookmark = self.setup_bookmark()
+
+        with sync_playwright() as p:
+            url = reverse("bookmarks:index") + f"?q={bookmark.title}"
+            self.open(url, p)
+
+            details_modal = self.open_details_modal(bookmark)
+            asset_list = details_modal.locator(".assets")
+
+            # No snapshots initially
+            snapshot = asset_list.get_by_text("HTML snapshot from", exact=False)
+            expect(snapshot).not_to_be_visible()
+
+            # Create snapshot
+            details_modal.get_by_text("Create HTML snapshot", exact=False).click()
+            self.assertReloads(0)
+
+            # Has new snapshots
+            expect(snapshot).to_be_visible()
+
+            # Create snapshot
+            asset_list.get_by_text("Remove", exact=False).click()
+            asset_list.get_by_text("Confirm", exact=False).click()
+
+            # Snapshot is removed
+            expect(snapshot).not_to_be_visible()
+            self.assertReloads(0)

bookmarks/e2e/e2e_test_bookmark_form.py

@@ -85,3 +85,25 @@ class BookmarkFormE2ETestCase(LinkdingE2ETestCase):
 
             page.get_by_label("URL").fill(bookmark.url)
             expect(details).to_have_attribute("open", value="")
+
+    def test_create_should_preview_auto_tags(self):
+        profile = self.get_or_create_test_user().profile
+        profile.auto_tagging_rules = "github.com dev github"
+        profile.save()
+
+        with sync_playwright() as p:
+            # Open page with URL that should have auto tags
+            browser = self.setup_browser(p)
+            page = browser.new_page()
+            url = self.live_server_url + reverse("bookmarks:new")
+            url += f"?url=https%3A%2F%2Fgithub.com%2Fsissbruecker%2Flinkding"
+            page.goto(url)
+
+            auto_tags_hint = page.locator(".form-input-hint.auto-tags")
+            expect(auto_tags_hint).to_be_visible()
+            expect(auto_tags_hint).to_have_text("Auto tags: dev github")
+
+            # Change to URL without auto tags
+            page.get_by_label("URL").fill("https://example.com")
+
+            expect(auto_tags_hint).to_be_hidden()

bookmarks/e2e/e2e_test_bookmark_page_bulk_edit.py

@@ -194,7 +194,7 @@ class BookmarkPagePartialUpdatesE2ETestCase(LinkdingE2ETestCase):
 
             self.locate_bulk_edit_toggle().click()
 
-            checkboxes = page.locator("label[ld-bulk-edit-checkbox] input")
+            checkboxes = page.locator("label.bulk-edit-checkbox input")
             self.assertEqual(6, checkboxes.count())
             for i in range(checkboxes.count()):
                 expect(checkboxes.nth(i)).not_to_be_checked()
@@ -264,13 +264,13 @@ class BookmarkPagePartialUpdatesE2ETestCase(LinkdingE2ETestCase):
 
             # Hide select across by toggling a single bookmark
             self.locate_bookmark("Bookmark 1").locator(
-                "label[ld-bulk-edit-checkbox]"
+                "label.bulk-edit-checkbox"
             ).click()
             expect(self.locate_bulk_edit_select_across()).not_to_be_visible()
 
             # Show select across again, verify it is unchecked
             self.locate_bookmark("Bookmark 1").locator(
-                "label[ld-bulk-edit-checkbox]"
+                "label.bulk-edit-checkbox"
             ).click()
             expect(self.locate_bulk_edit_select_across()).not_to_be_checked()
 
@@ -297,7 +297,7 @@ class BookmarkPagePartialUpdatesE2ETestCase(LinkdingE2ETestCase):
             expect(bookmark_list).not_to_be_visible()
 
             # Verify bulk edit checkboxes are reset
-            checkboxes = page.locator("label[ld-bulk-edit-checkbox] input")
+            checkboxes = page.locator("label.bulk-edit-checkbox input")
             self.assertEqual(31, checkboxes.count())
             for i in range(checkboxes.count()):
                 expect(checkboxes.nth(i)).not_to_be_checked()

bookmarks/e2e/e2e_test_bookmark_page_partial_updates.py

@@ -169,7 +169,7 @@ class BookmarkPagePartialUpdatesE2ETestCase(LinkdingE2ETestCase):
 
             self.locate_bulk_edit_toggle().click()
             self.locate_bookmark("Bookmark 2").locator(
-                "label[ld-bulk-edit-checkbox]"
+                "label.bulk-edit-checkbox"
             ).click()
             self.select_bulk_action("Archive")
             self.locate_bulk_edit_bar().get_by_text("Execute").click()
@@ -187,7 +187,7 @@ class BookmarkPagePartialUpdatesE2ETestCase(LinkdingE2ETestCase):
 
             self.locate_bulk_edit_toggle().click()
             self.locate_bookmark("Bookmark 2").locator(
-                "label[ld-bulk-edit-checkbox]"
+                "label.bulk-edit-checkbox"
             ).click()
             self.select_bulk_action("Delete")
             self.locate_bulk_edit_bar().get_by_text("Execute").click()
@@ -230,7 +230,7 @@ class BookmarkPagePartialUpdatesE2ETestCase(LinkdingE2ETestCase):
 
             self.locate_bulk_edit_toggle().click()
             self.locate_bookmark("Archived Bookmark 2").locator(
-                "label[ld-bulk-edit-checkbox]"
+                "label.bulk-edit-checkbox"
             ).click()
             self.select_bulk_action("Unarchive")
             self.locate_bulk_edit_bar().get_by_text("Execute").click()
@@ -248,7 +248,7 @@ class BookmarkPagePartialUpdatesE2ETestCase(LinkdingE2ETestCase):
 
             self.locate_bulk_edit_toggle().click()
             self.locate_bookmark("Archived Bookmark 2").locator(
-                "label[ld-bulk-edit-checkbox]"
+                "label.bulk-edit-checkbox"
             ).click()
             self.select_bulk_action("Delete")
             self.locate_bulk_edit_bar().get_by_text("Execute").click()

bookmarks/e2e/e2e_test_tag_cloud_modal.py

@@ -0,0 +1,76 @@
+from django.test import override_settings
+from django.urls import reverse
+from playwright.sync_api import sync_playwright, expect, Locator
+
+from bookmarks.e2e.helpers import LinkdingE2ETestCase
+from bookmarks.models import Bookmark
+
+
+class TagCloudModalE2ETestCase(LinkdingE2ETestCase):
+    def test_show_modal_close_modal(self):
+        self.setup_bookmark(tags=[self.setup_tag(name="cooking")])
+        self.setup_bookmark(tags=[self.setup_tag(name="hiking")])
+
+        with sync_playwright() as p:
+            page = self.open(reverse("bookmarks:index"), p)
+
+            # use smaller viewport to make tags button visible
+            page.set_viewport_size({"width": 375, "height": 812})
+
+            # open tag cloud modal
+            modal_trigger = page.locator(".content-area-header").get_by_role(
+                "button", name="Tags"
+            )
+            modal_trigger.click()
+
+            # verify modal is visible
+            modal = page.locator(".modal")
+            expect(modal).to_be_visible()
+            expect(modal.locator(".modal-title")).to_have_text("Tags")
+
+            # close with close button
+            modal.locator("button.close").click()
+            expect(modal).to_be_hidden()
+
+            # open modal again
+            modal_trigger.click()
+
+            # close with backdrop
+            backdrop = modal.locator(".modal-overlay")
+            backdrop.click(position={"x": 0, "y": 0})
+            expect(modal).to_be_hidden()
+
+    def test_select_tag(self):
+        self.setup_bookmark(tags=[self.setup_tag(name="cooking")])
+        self.setup_bookmark(tags=[self.setup_tag(name="hiking")])
+
+        with sync_playwright() as p:
+            page = self.open(reverse("bookmarks:index"), p)
+
+            # use smaller viewport to make tags button visible
+            page.set_viewport_size({"width": 375, "height": 812})
+
+            # open tag cloud modal
+            modal_trigger = page.locator(".content-area-header").get_by_role(
+                "button", name="Tags"
+            )
+            modal_trigger.click()
+
+            # verify tags are displayed
+            modal = page.locator(".modal")
+            unselected_tags = modal.locator(".unselected-tags")
+            expect(unselected_tags.get_by_text("cooking")).to_be_visible()
+            expect(unselected_tags.get_by_text("hiking")).to_be_visible()
+
+            # select tag
+            unselected_tags.get_by_text("cooking").click()
+
+            # open modal again
+            modal_trigger.click()
+
+            # verify tag is selected, other tag is not visible anymore
+            selected_tags = modal.locator(".selected-tags")
+            expect(selected_tags.get_by_text("cooking")).to_be_visible()
+
+            expect(unselected_tags.get_by_text("cooking")).not_to_be_visible()
+            expect(unselected_tags.get_by_text("hiking")).not_to_be_visible()

bookmarks/e2e/helpers.py

@@ -39,6 +39,9 @@ class LinkdingE2ETestCase(LiveServerTestCase, BookmarkFactoryMixin):
     def assertReloads(self, count: int):
         self.assertEqual(self.num_loads, count)
 
+    def resetReloads(self):
+        self.num_loads = 0
+
     def locate_bookmark_list(self):
         return self.page.locator("ul[ld-bookmark-list]")
 
@@ -62,7 +65,7 @@ class LinkdingE2ETestCase(LiveServerTestCase, BookmarkFactoryMixin):
         return self.page.locator(".bulk-edit-bar")
 
     def locate_bulk_edit_select_all(self):
-        return self.locate_bulk_edit_bar().locator("label[ld-bulk-edit-checkbox][all]")
+        return self.locate_bulk_edit_bar().locator("label.bulk-edit-checkbox.all")
 
     def locate_bulk_edit_select_across(self):
         return self.locate_bulk_edit_bar().locator("label.select-across")

bookmarks/feeds.py

@@ -2,7 +2,8 @@ import unicodedata
 from dataclasses import dataclass
 
 from django.contrib.syndication.views import Feed
-from django.db.models import QuerySet
+from django.db.models import QuerySet, prefetch_related_objects
+from django.http import HttpRequest
 from django.urls import reverse
 
 from bookmarks import queries
@@ -11,6 +12,7 @@ from bookmarks.models import Bookmark, BookmarkSearch, FeedToken, UserProfile
 
 @dataclass
 class FeedContext:
+    request: HttpRequest
     feed_token: FeedToken | None
     query_set: QuerySet[Bookmark]
 
@@ -26,13 +28,27 @@ def sanitize(text: str):
 
 
 class BaseBookmarksFeed(Feed):
-    def get_object(self, request, feed_key: str):
+    def get_object(self, request, feed_key: str | None):
-        feed_token = FeedToken.objects.get(key__exact=feed_key)
+        feed_token = FeedToken.objects.get(key__exact=feed_key) if feed_key else None
-        search = BookmarkSearch(q=request.GET.get("q", ""))
+        search = BookmarkSearch(
-        query_set = queries.query_bookmarks(
+            q=request.GET.get("q", ""),
-            feed_token.user, feed_token.user.profile, search
+            unread=request.GET.get("unread", ""),
+            shared=request.GET.get("shared", ""),
         )
-        return FeedContext(feed_token, query_set)
+        query_set = self.get_query_set(feed_token, search)
+        return FeedContext(request, feed_token, query_set)
+
+    def get_query_set(self, feed_token: FeedToken, search: BookmarkSearch):
+        raise NotImplementedError
+
+    def items(self, context: FeedContext):
+        limit = context.request.GET.get("limit", 100)
+        if limit:
+            data = context.query_set[: int(limit)]
+        else:
+            data = list(context.query_set)
+        prefetch_related_objects(data, "tags")
+        return data
 
     def item_title(self, item: Bookmark):
         return sanitize(item.resolved_title)
@@ -46,60 +62,56 @@ class BaseBookmarksFeed(Feed):
     def item_pubdate(self, item: Bookmark):
         return item.date_added
 
+    def item_categories(self, item: Bookmark):
+        return item.tag_names
+
 
 class AllBookmarksFeed(BaseBookmarksFeed):
     title = "All bookmarks"
     description = "All bookmarks"
 
+    def get_query_set(self, feed_token: FeedToken, search: BookmarkSearch):
+        return queries.query_bookmarks(feed_token.user, feed_token.user.profile, search)
+
     def link(self, context: FeedContext):
         return reverse("bookmarks:feeds.all", args=[context.feed_token.key])
 
-    def items(self, context: FeedContext):
-        return context.query_set
-
 
 class UnreadBookmarksFeed(BaseBookmarksFeed):
     title = "Unread bookmarks"
     description = "All unread bookmarks"
 
+    def get_query_set(self, feed_token: FeedToken, search: BookmarkSearch):
+        return queries.query_bookmarks(
+            feed_token.user, feed_token.user.profile, search
+        ).filter(unread=True)
+
     def link(self, context: FeedContext):
         return reverse("bookmarks:feeds.unread", args=[context.feed_token.key])
 
-    def items(self, context: FeedContext):
-        return context.query_set.filter(unread=True)
-
 
 class SharedBookmarksFeed(BaseBookmarksFeed):
     title = "Shared bookmarks"
     description = "All shared bookmarks"
 
-    def get_object(self, request, feed_key: str):
+    def get_query_set(self, feed_token: FeedToken, search: BookmarkSearch):
-        feed_token = FeedToken.objects.get(key__exact=feed_key)
+        return queries.query_shared_bookmarks(
-        search = BookmarkSearch(q=request.GET.get("q", ""))
-        query_set = queries.query_shared_bookmarks(
             None, feed_token.user.profile, search, False
         )
-        return FeedContext(feed_token, query_set)
 
     def link(self, context: FeedContext):
         return reverse("bookmarks:feeds.shared", args=[context.feed_token.key])
 
-    def items(self, context: FeedContext):
-        return context.query_set
-
 
 class PublicSharedBookmarksFeed(BaseBookmarksFeed):
     title = "Public shared bookmarks"
     description = "All public shared bookmarks"
 
     def get_object(self, request):
-        search = BookmarkSearch(q=request.GET.get("q", ""))
+        return super().get_object(request, None)
-        default_profile = UserProfile()
+
-        query_set = queries.query_shared_bookmarks(None, default_profile, search, True)
+    def get_query_set(self, feed_token: FeedToken, search: BookmarkSearch):
-        return FeedContext(None, query_set)
+        return queries.query_shared_bookmarks(None, UserProfile(), search, True)
 
     def link(self, context: FeedContext):
         return reverse("bookmarks:feeds.public_shared")
-
-    def items(self, context: FeedContext):
-        return context.query_set

bookmarks/frontend/behaviors/bookmark-details.js

@@ -1,38 +0,0 @@
-import { registerBehavior } from "./index";
-
-class BookmarkDetails {
-  constructor(element) {
-    this.form = element.querySelector(".status form");
-    if (!this.form) {
-      // Form may not exist if user does not own the bookmark
-      return;
-    }
-    this.form.addEventListener("submit", (event) => {
-      event.preventDefault();
-      this.submitForm();
-    });
-
-    const inputs = this.form.querySelectorAll("input");
-    inputs.forEach((input) => {
-      input.addEventListener("change", () => {
-        this.submitForm();
-      });
-    });
-  }
-
-  async submitForm() {
-    const url = this.form.action;
-    const formData = new FormData(this.form);
-
-    await fetch(url, {
-      method: "POST",
-      body: formData,
-      redirect: "manual", // ignore redirect
-    });
-
-    // Refresh bookmark page if it exists
-    document.dispatchEvent(new CustomEvent("bookmark-page-refresh"));
-  }
-}
-
-registerBehavior("ld-bookmark-details", BookmarkDetails);

bookmarks/frontend/behaviors/bookmark-page.js

@@ -1,67 +1,8 @@
-import { registerBehavior, swap } from "./index";
+import { Behavior, registerBehavior } from "./index";
 
-class BookmarkPage {
+class BookmarkItem extends Behavior {
   constructor(element) {
-    this.element = element;
+    super(element);
-    this.form = element.querySelector("form.bookmark-actions");
-    this.form.addEventListener("submit", this.onFormSubmit.bind(this));
-
-    this.bookmarkList = element.querySelector(".bookmark-list-container");
-    this.tagCloud = element.querySelector(".tag-cloud-container");
-
-    document.addEventListener("bookmark-page-refresh", () => {
-      this.refresh();
-    });
-  }
-
-  async onFormSubmit(event) {
-    event.preventDefault();
-
-    const url = this.form.action;
-    const formData = new FormData(this.form);
-    formData.append(event.submitter.name, event.submitter.value);
-
-    await fetch(url, {
-      method: "POST",
-      body: formData,
-      redirect: "manual", // ignore redirect
-    });
-    await this.refresh();
-  }
-
-  async refresh() {
-    const query = window.location.search;
-    const bookmarksUrl = this.element.getAttribute("bookmarks-url");
-    const tagsUrl = this.element.getAttribute("tags-url");
-    Promise.all([
-      fetch(`${bookmarksUrl}${query}`).then((response) => response.text()),
-      fetch(`${tagsUrl}${query}`).then((response) => response.text()),
-    ]).then(([bookmarkListHtml, tagCloudHtml]) => {
-      swap(this.bookmarkList, bookmarkListHtml);
-      swap(this.tagCloud, tagCloudHtml);
-
-      // Dispatch list updated event
-      const listElement = this.bookmarkList.querySelector(
-        "ul[data-bookmarks-total]",
-      );
-      const bookmarksTotal =
-        (listElement && listElement.dataset.bookmarksTotal) || 0;
-
-      this.bookmarkList.dispatchEvent(
-        new CustomEvent("bookmark-list-updated", {
-          bubbles: true,
-          detail: { bookmarksTotal },
-        }),
-      );
-    });
-  }
-}
-
-registerBehavior("ld-bookmark-page", BookmarkPage);
-
-class BookmarkItem {
-  constructor(element) {
-    this.element = element;
 
     // Toggle notes
     const notesToggle = element.querySelector(".toggle-notes");
@@ -72,9 +13,11 @@ class BookmarkItem {
     // Add tooltip to title if it is truncated
     const titleAnchor = element.querySelector(".title > a");
     const titleSpan = titleAnchor.querySelector("span");
-    if (titleSpan.offsetWidth > titleAnchor.offsetWidth) {
+    requestAnimationFrame(() => {
-      titleAnchor.dataset.tooltip = titleSpan.textContent;
+      if (titleSpan.offsetWidth > titleAnchor.offsetWidth) {
-    }
+        titleAnchor.dataset.tooltip = titleSpan.textContent;
+      }
+    });
   }
 
   onToggleNotes(event) {

bookmarks/frontend/behaviors/bulk-edit.js

@@ -1,46 +1,60 @@
-import { registerBehavior } from "./index";
+import { Behavior, registerBehavior } from "./index";
 
-class BulkEdit {
+class BulkEdit extends Behavior {
   constructor(element) {
-    this.element = element;
+    super(element);
+
     this.active = false;
-    this.actionSelect = element.querySelector("select[name='bulk_action']");
-    this.tagAutoComplete = element.querySelector(".tag-autocomplete");
-    this.selectAcross = element.querySelector("label.select-across");
 
-    element.addEventListener(
+    this.onToggleActive = this.onToggleActive.bind(this);
-      "bulk-edit-toggle-active",
+    this.onToggleAll = this.onToggleAll.bind(this);
-      this.onToggleActive.bind(this),
+    this.onToggleBookmark = this.onToggleBookmark.bind(this);
-    );
+    this.onActionSelected = this.onActionSelected.bind(this);
-    element.addEventListener(
-      "bulk-edit-toggle-all",
-      this.onToggleAll.bind(this),
-    );
-    element.addEventListener(
-      "bulk-edit-toggle-bookmark",
-      this.onToggleBookmark.bind(this),
-    );
-    element.addEventListener(
-      "bookmark-list-updated",
-      this.onListUpdated.bind(this),
-    );
 
-    this.actionSelect.addEventListener(
+    this.init();
-      "change",
+    // Reset when bookmarks are refreshed
-      this.onActionSelected.bind(this),
+    document.addEventListener("refresh-bookmark-list-done", () => this.init());
-    );
   }
 
-  get allCheckbox() {
+  init() {
-    return this.element.querySelector("[ld-bulk-edit-checkbox][all] input");
+    // Update elements
-  }
+    this.activeToggle = this.element.querySelector(".bulk-edit-active-toggle");
+    this.actionSelect = this.element.querySelector(
+      "select[name='bulk_action']",
+    );
+    this.tagAutoComplete = this.element.querySelector(".tag-autocomplete");
+    this.selectAcross = this.element.querySelector("label.select-across");
+    this.allCheckbox = this.element.querySelector(
+      ".bulk-edit-checkbox.all input",
+    );
+    this.bookmarkCheckboxes = Array.from(
+      this.element.querySelectorAll(".bulk-edit-checkbox:not(.all) input"),
+    );
 
-  get bookmarkCheckboxes() {
+    // Remove previous listeners if elements are the same
-    return [
+    this.activeToggle.removeEventListener("click", this.onToggleActive);
-      ...this.element.querySelectorAll(
+    this.actionSelect.removeEventListener("change", this.onActionSelected);
-        "[ld-bulk-edit-checkbox]:not([all]) input",
+    this.allCheckbox.removeEventListener("change", this.onToggleAll);
-      ),
+    this.bookmarkCheckboxes.forEach((checkbox) => {
-    ];
+      checkbox.removeEventListener("change", this.onToggleBookmark);
+    });
+
+    // Reset checkbox states
+    this.reset();
+
+    // Update total number of bookmarks
+    const totalHolder = this.element.querySelector("[data-bookmarks-total]");
+    const total = totalHolder?.dataset.bookmarksTotal || 0;
+    const totalSpan = this.selectAcross.querySelector("span.total");
+    totalSpan.textContent = total;
+
+    // Add new listeners
+    this.activeToggle.addEventListener("click", this.onToggleActive);
+    this.actionSelect.addEventListener("change", this.onActionSelected);
+    this.allCheckbox.addEventListener("change", this.onToggleAll);
+    this.bookmarkCheckboxes.forEach((checkbox) => {
+      checkbox.addEventListener("change", this.onToggleBookmark);
+    });
   }
 
   onToggleActive() {
@@ -81,16 +95,6 @@ class BulkEdit {
     }
   }
 
-  onListUpdated(event) {
-    // Reset checkbox states
-    this.reset();
-
-    // Update total number of bookmarks
-    const total = event.detail.bookmarksTotal;
-    const totalSpan = this.selectAcross.querySelector("span.total");
-    totalSpan.textContent = total;
-  }
-
   updateSelectAcross(allChecked) {
     if (allChecked) {
       this.selectAcross.classList.remove("d-none");
@@ -109,33 +113,4 @@ class BulkEdit {
   }
 }
 
-class BulkEditActiveToggle {
-  constructor(element) {
-    this.element = element;
-    element.addEventListener("click", this.onClick.bind(this));
-  }
-
-  onClick() {
-    this.element.dispatchEvent(
-      new CustomEvent("bulk-edit-toggle-active", { bubbles: true }),
-    );
-  }
-}
-
-class BulkEditCheckbox {
-  constructor(element) {
-    this.element = element;
-    element.addEventListener("change", this.onChange.bind(this));
-  }
-
-  onChange() {
-    const type = this.element.hasAttribute("all") ? "all" : "bookmark";
-    this.element.dispatchEvent(
-      new CustomEvent(`bulk-edit-toggle-${type}`, { bubbles: true }),
-    );
-  }
-}
-
 registerBehavior("ld-bulk-edit", BulkEdit);
-registerBehavior("ld-bulk-edit-active-toggle", BulkEditActiveToggle);
-registerBehavior("ld-bulk-edit-checkbox", BulkEditCheckbox);

bookmarks/frontend/behaviors/confirm-button.js

@@ -1,25 +1,29 @@
-import { registerBehavior } from "./index";
+import { Behavior, registerBehavior } from "./index";
 
-class ConfirmButtonBehavior {
+class ConfirmButtonBehavior extends Behavior {
   constructor(element) {
-    const button = element;
+    super(element);
-    button.dataset.type = button.type;
+    element.dataset.type = element.type;
-    button.dataset.name = button.name;
+    element.dataset.name = element.name;
-    button.dataset.value = button.value;
+    element.dataset.value = element.value;
-    button.removeAttribute("type");
+    element.removeAttribute("type");
-    button.removeAttribute("name");
+    element.removeAttribute("name");
-    button.removeAttribute("value");
+    element.removeAttribute("value");
-    button.addEventListener("click", this.onClick.bind(this));
+    element.addEventListener("click", this.onClick.bind(this));
-    this.button = button;
+  }
+
+  destroy() {
+    Behavior.interacting = false;
   }
 
   onClick(event) {
     event.preventDefault();
+    Behavior.interacting = true;
 
     const container = document.createElement("span");
     container.className = "confirmation";
 
-    const icon = this.button.getAttribute("confirm-icon");
+    const icon = this.element.getAttribute("ld-confirm-icon");
     if (icon) {
       const iconElement = document.createElementNS(
         "http://www.w3.org/2000/svg",
@@ -31,27 +35,27 @@ class ConfirmButtonBehavior {
       container.append(iconElement);
     }
 
-    const question = this.button.getAttribute("confirm-question");
+    const question = this.element.getAttribute("ld-confirm-question");
     if (question) {
       const questionElement = document.createElement("span");
       questionElement.innerText = question;
       container.append(question);
     }
 
-    const buttonClasses = Array.from(this.button.classList.values())
+    const buttonClasses = Array.from(this.element.classList.values())
       .filter((cls) => cls.startsWith("btn"))
       .join(" ");
 
-    const cancelButton = document.createElement(this.button.nodeName);
+    const cancelButton = document.createElement(this.element.nodeName);
     cancelButton.type = "button";
     cancelButton.innerText = question ? "No" : "Cancel";
     cancelButton.className = `${buttonClasses} mr-1`;
     cancelButton.addEventListener("click", this.reset.bind(this));
 
-    const confirmButton = document.createElement(this.button.nodeName);
+    const confirmButton = document.createElement(this.element.nodeName);
-    confirmButton.type = this.button.dataset.type;
+    confirmButton.type = this.element.dataset.type;
-    confirmButton.name = this.button.dataset.name;
+    confirmButton.name = this.element.dataset.name;
-    confirmButton.value = this.button.dataset.value;
+    confirmButton.value = this.element.dataset.value;
     confirmButton.innerText = question ? "Yes" : "Confirm";
     confirmButton.className = buttonClasses;
     confirmButton.addEventListener("click", this.reset.bind(this));
@@ -59,14 +63,15 @@ class ConfirmButtonBehavior {
     container.append(cancelButton, confirmButton);
     this.container = container;
 
-    this.button.before(container);
+    this.element.before(container);
-    this.button.classList.add("d-none");
+    this.element.classList.add("d-none");
   }
 
   reset() {
     setTimeout(() => {
+      Behavior.interacting = false;
       this.container.remove();
-      this.button.classList.remove("d-none");
+      this.element.classList.remove("d-none");
     });
   }
 }

bookmarks/frontend/behaviors/dropdown.js

@@ -1,8 +1,8 @@
-import { registerBehavior } from "./index";
+import { Behavior, registerBehavior } from "./index";
 
-class DropdownBehavior {
+class DropdownBehavior extends Behavior {
   constructor(element) {
-    this.element = element;
+    super(element);
     this.opened = false;
     this.onOutsideClick = this.onOutsideClick.bind(this);
 

bookmarks/frontend/behaviors/fetch.js

@@ -0,0 +1,48 @@
+import { Behavior, fireEvents, registerBehavior, swap } from "./index";
+
+class FetchBehavior extends Behavior {
+  constructor(element) {
+    super(element);
+
+    const eventName = element.getAttribute("ld-on");
+    const interval = parseInt(element.getAttribute("ld-interval")) * 1000;
+
+    this.onFetch = this.onFetch.bind(this);
+    this.onInterval = this.onInterval.bind(this);
+
+    element.addEventListener(eventName, this.onFetch);
+    if (interval) {
+      this.intervalId = setInterval(this.onInterval, interval);
+    }
+  }
+
+  destroy() {
+    if (this.intervalId) {
+      clearInterval(this.intervalId);
+    }
+  }
+
+  async onFetch(maybeEvent) {
+    if (maybeEvent) {
+      maybeEvent.preventDefault();
+    }
+    const url = this.element.getAttribute("ld-fetch");
+    const html = await fetch(url).then((response) => response.text());
+
+    const target = this.element.getAttribute("ld-target");
+    const select = this.element.getAttribute("ld-select");
+    swap(this.element, html, { target, select });
+
+    const events = this.element.getAttribute("ld-fire");
+    fireEvents(events);
+  }
+
+  onInterval() {
+    if (Behavior.interacting) {
+      return;
+    }
+    this.onFetch();
+  }
+}
+
+registerBehavior("ld-fetch", FetchBehavior);

bookmarks/frontend/behaviors/form.js

@@ -0,0 +1,64 @@
+import { Behavior, fireEvents, registerBehavior } from "./index";
+
+class FormBehavior extends Behavior {
+  constructor(element) {
+    super(element);
+
+    element.addEventListener("submit", this.onSubmit.bind(this));
+  }
+
+  async onSubmit(event) {
+    event.preventDefault();
+
+    const url = this.element.action;
+    const formData = new FormData(this.element);
+    if (event.submitter) {
+      formData.append(event.submitter.name, event.submitter.value);
+    }
+
+    await fetch(url, {
+      method: "POST",
+      body: formData,
+      redirect: "manual", // ignore redirect
+    });
+
+    const events = this.element.getAttribute("ld-fire");
+    if (fireEvents) {
+      fireEvents(events);
+    }
+  }
+}
+
+class AutoSubmitBehavior extends Behavior {
+  constructor(element) {
+    super(element);
+
+    element.addEventListener("change", () => {
+      const form = element.closest("form");
+      form.dispatchEvent(new Event("submit", { cancelable: true }));
+    });
+  }
+}
+
+class UploadButton extends Behavior {
+  constructor(element) {
+    super(element);
+
+    const fileInput = element.nextElementSibling;
+
+    element.addEventListener("click", () => {
+      fileInput.click();
+    });
+
+    fileInput.addEventListener("change", () => {
+      const form = fileInput.closest("form");
+      const event = new Event("submit", { cancelable: true });
+      event.submitter = element;
+      form.dispatchEvent(event);
+    });
+  }
+}
+
+registerBehavior("ld-form", FormBehavior);
+registerBehavior("ld-auto-submit", AutoSubmitBehavior);
+registerBehavior("ld-upload-button", UploadButton);

bookmarks/frontend/behaviors/global-shortcuts.js

@@ -1,7 +1,9 @@
-import { registerBehavior } from "./index";
+import { Behavior, registerBehavior } from "./index";
+
+class GlobalShortcuts extends Behavior {
+  constructor(element) {
+    super(element);
 
-class GlobalShortcuts {
-  constructor() {
     document.addEventListener("keydown", this.onKeyDown.bind(this));
   }
 

bookmarks/frontend/behaviors/index.js

@@ -1,4 +1,35 @@
 const behaviorRegistry = {};
+const debug = false;
+
+const mutationObserver = new MutationObserver((mutations) => {
+  mutations.forEach((mutation) => {
+    mutation.removedNodes.forEach((node) => {
+      if (node instanceof HTMLElement && !node.isConnected) {
+        destroyBehaviors(node);
+      }
+    });
+    mutation.addedNodes.forEach((node) => {
+      if (node instanceof HTMLElement && node.isConnected) {
+        applyBehaviors(node);
+      }
+    });
+  });
+});
+
+mutationObserver.observe(document.body, {
+  childList: true,
+  subtree: true,
+});
+
+export class Behavior {
+  constructor(element) {
+    this.element = element;
+  }
+
+  destroy() {}
+}
+
+Behavior.interacting = false;
 
 export function registerBehavior(name, behavior) {
   behaviorRegistry[name] = behavior;
@@ -12,7 +43,14 @@ export function applyBehaviors(container, behaviorNames = null) {
 
   behaviorNames.forEach((behaviorName) => {
     const behavior = behaviorRegistry[behaviorName];
-    const elements = container.querySelectorAll(`[${behaviorName}]`);
+    const elements = Array.from(
+      container.querySelectorAll(`[${behaviorName}]`),
+    );
+
+    // Include the container element if it has the behavior
+    if (container.hasAttribute && container.hasAttribute(behaviorName)) {
+      elements.push(container);
+    }
 
     elements.forEach((element) => {
       element.__behaviors = element.__behaviors || [];
@@ -26,11 +64,82 @@ export function applyBehaviors(container, behaviorNames = null) {
 
       const behaviorInstance = new behavior(element);
       element.__behaviors.push(behaviorInstance);
+      if (debug) {
+        console.log(
+          `[Behavior] ${behaviorInstance.constructor.name} initialized`,
+        );
+      }
     });
   });
 }
 
-export function swap(element, html) {
+export function destroyBehaviors(element) {
-  element.innerHTML = html;
+  const behaviorNames = Object.keys(behaviorRegistry);
-  applyBehaviors(element);
+
+  behaviorNames.forEach((behaviorName) => {
+    const elements = Array.from(element.querySelectorAll(`[${behaviorName}]`));
+    elements.push(element);
+
+    elements.forEach((element) => {
+      if (!element.__behaviors) {
+        return;
+      }
+
+      element.__behaviors.forEach((behavior) => {
+        behavior.destroy();
+        if (debug) {
+          console.log(`[Behavior] ${behavior.constructor.name} destroyed`);
+        }
+      });
+      delete element.__behaviors;
+    });
+  });
+}
+
+export function swap(element, html, options) {
+  const dom = new DOMParser().parseFromString(html, "text/html");
+
+  let targetElement = element;
+  let strategy = "innerHTML";
+  if (options.target) {
+    const parts = options.target.split("|");
+    targetElement =
+      parts[0] === "self" ? element : document.querySelector(parts[0]);
+    strategy = parts[1] || "innerHTML";
+  }
+
+  let contents = Array.from(dom.body.children);
+  if (options.select) {
+    contents = Array.from(dom.querySelectorAll(options.select));
+  }
+
+  switch (strategy) {
+    case "append":
+      targetElement.append(...contents);
+      break;
+    case "outerHTML":
+      targetElement.parentElement.replaceChild(contents[0], targetElement);
+      break;
+    case "innerHTML":
+    default:
+      Array.from(targetElement.children).forEach((child) => {
+        child.remove();
+      });
+      targetElement.append(...contents);
+  }
+}
+
+export function fireEvents(events) {
+  if (!events) {
+    return;
+  }
+  events.split(",").forEach((eventName) => {
+    const targets = Array.from(
+      document.querySelectorAll(`[ld-on='${eventName}']`),
+    );
+    targets.push(document);
+    targets.forEach((target) => {
+      target.dispatchEvent(new CustomEvent(eventName));
+    });
+  });
 }

bookmarks/frontend/behaviors/modal.js

@@ -1,97 +1,48 @@
-import { applyBehaviors, registerBehavior } from "./index";
+import { Behavior, registerBehavior } from "./index";
 
-class ModalBehavior {
+class ModalBehavior extends Behavior {
   constructor(element) {
-    const toggle = element;
+    super(element);
-    toggle.addEventListener("click", this.onToggleClick.bind(this));
+
-    this.toggle = toggle;
+    this.onClose = this.onClose.bind(this);
+    this.onKeyDown = this.onKeyDown.bind(this);
+
+    const modalOverlay = element.querySelector(".modal-overlay");
+    const closeButton = element.querySelector("button.close");
+    modalOverlay.addEventListener("click", this.onClose);
+    closeButton.addEventListener("click", this.onClose);
+
+    document.addEventListener("keydown", this.onKeyDown);
   }
 
-  async onToggleClick(event) {
+  destroy() {
-    // Ignore Ctrl + click
+    document.removeEventListener("keydown", this.onKeyDown);
-    if (event.ctrlKey || event.metaKey) {
+  }
-      return;
-    }
-    event.preventDefault();
-    event.stopPropagation();
 
-    // Create modal either by teleporting existing content or fetching from URL
+  onKeyDown(event) {
-    const modal = this.toggle.hasAttribute("modal-content")
+    // Skip if event occurred within an input element
-      ? this.createFromContent()
+    const targetNodeName = event.target.nodeName;
-      : await this.createFromUrl();
+    const isInputTarget =
+      targetNodeName === "INPUT" ||
+      targetNodeName === "SELECT" ||
+      targetNodeName === "TEXTAREA";
 
-    if (!modal) {
+    if (isInputTarget) {
       return;
     }
 
-    // Register close handlers
+    if (event.key === "Escape") {
-    const modalOverlay = modal.querySelector(".modal-overlay");
+      event.preventDefault();
-    const closeButton = modal.querySelector("button.close");
+      this.onClose();
-    modalOverlay.addEventListener("click", this.onClose.bind(this));
-    closeButton.addEventListener("click", this.onClose.bind(this));
-
-    document.body.append(modal);
-    applyBehaviors(document.body);
-    this.modal = modal;
-  }
-
-  async createFromUrl() {
-    const url = this.toggle.getAttribute("modal-url");
-    const modalHtml = await fetch(url).then((response) => response.text());
-    const parser = new DOMParser();
-    const doc = parser.parseFromString(modalHtml, "text/html");
-    return doc.querySelector(".modal");
-  }
-
-  createFromContent() {
-    const contentSelector = this.toggle.getAttribute("modal-content");
-    const content = document.querySelector(contentSelector);
-    if (!content) {
-      return;
     }
-
-    // Todo: make title configurable, only used for tag cloud for now
-    const modal = document.createElement("div");
-    modal.classList.add("modal", "active");
-    modal.innerHTML = `
-      <div class="modal-overlay" aria-label="Close"></div>
-      <div class="modal-container">
-        <div class="modal-header d-flex justify-between align-center">
-          <div class="modal-title h5">Tags</div>
-          <button class="close">
-          <svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" stroke-width="2" stroke="currentColor" fill="none" stroke-linecap="round" stroke-linejoin="round">
-            <path stroke="none" d="M0 0h24v24H0z" fill="none"></path>
-            <path d="M18 6l-12 12"></path>
-            <path d="M6 6l12 12"></path>
-          </svg>
-          </button>
-        </div>
-        <div class="modal-body">
-          <div class="content"></div>
-        </div>
-      </div>
-    `;
-
-    const contentOwner = content.parentElement;
-    const contentContainer = modal.querySelector(".content");
-    contentContainer.append(content);
-    this.content = content;
-    this.contentOwner = contentOwner;
-
-    return modal;
   }
 
   onClose() {
-    // Teleport content back
+    document.removeEventListener("keydown", this.onKeyDown);
-    if (this.content && this.contentOwner) {
+    this.element.classList.add("closing");
-      this.contentOwner.append(this.content);
+    this.element.addEventListener("animationend", (event) => {
-    }
-
-    // Remove modal
-    this.modal.classList.add("closing");
-    this.modal.addEventListener("animationend", (event) => {
       if (event.animationName === "fade-out") {
-        this.modal.remove();
+        this.element.remove();
       }
     });
   }

bookmarks/frontend/behaviors/tag-autocomplete.js

@@ -1,9 +1,10 @@
-import { registerBehavior } from "./index";
+import { Behavior, registerBehavior } from "./index";
 import TagAutoCompleteComponent from "../components/TagAutocomplete.svelte";
 import { ApiClient } from "../api";
 
-class TagAutocomplete {
+class TagAutocomplete extends Behavior {
   constructor(element) {
+    super(element);
     const wrapper = document.createElement("div");
     const apiBaseUrl = document.documentElement.dataset.apiBaseUrl || "";
     const apiClient = new ApiClient(apiBaseUrl);

bookmarks/frontend/components/TagAutocomplete.svelte

@@ -150,17 +150,27 @@
         display: block;
     }
 
+    .form-autocomplete-input {
+        box-sizing: border-box;
+        height: var(--control-size);
+        min-height: var(--control-size);
+        padding: 0;
+    }
+
+    .form-autocomplete-input input {
+        width: 100%;
+        height: 100%;
+        border: none;
+        margin: 0;
+    }
+
     .form-autocomplete.small .form-autocomplete-input {
         height: var(--control-size-sm);
         min-height: var(--control-size-sm);
-        padding: 0.05rem 0.3rem;
     }
 
     .form-autocomplete.small .form-autocomplete-input input {
-        width: 100%;
+        padding: 0.05rem 0.3rem;
-        height: 100%;
-        margin: 0;
-        padding: 0;
         font-size: var(--font-size-sm);
     }
 

bookmarks/frontend/index.js

@@ -1,8 +1,9 @@
-import "./behaviors/bookmark-details";
 import "./behaviors/bookmark-page";
 import "./behaviors/bulk-edit";
 import "./behaviors/confirm-button";
 import "./behaviors/dropdown";
+import "./behaviors/fetch";
+import "./behaviors/form";
 import "./behaviors/modal";
 import "./behaviors/global-shortcuts";
 import "./behaviors/tag-autocomplete";

bookmarks/management/commands/backup.py

@@ -24,3 +24,8 @@ class Command(BaseCommand):
         source_db.close()
 
         self.stdout.write(self.style.SUCCESS(f"Backup created at {destination}"))
+        self.stdout.write(
+            self.style.WARNING(
+                "This backup method is deprecated and may be removed in the future. Please use the full_backup command instead, which creates backup zip file with all contents of the data folder."
+            )
+        )

bookmarks/management/commands/clean_tasks.py

@@ -1,15 +0,0 @@
-from background_task.models import Task, CompletedTask
-from django.core.management.base import BaseCommand
-
-
-class Command(BaseCommand):
-    help = "Remove task locks and clear completed task history"
-
-    def handle(self, *args, **options):
-        # Remove task locks
-        # If the background task processor exited while executing tasks, these tasks would still be marked as locked,
-        # even though no process is working on them, and would prevent the task processor from picking the next task in
-        # the queue
-        Task.objects.all().update(locked_by=None, locked_at=None)
-        # Clear task history to prevent them from bloating the DB
-        CompletedTask.objects.all().delete()

bookmarks/management/commands/full_backup.py

@@ -0,0 +1,75 @@
+import sqlite3
+import os
+import tempfile
+import zipfile
+
+from django.core.management.base import BaseCommand
+
+
+class Command(BaseCommand):
+    help = "Creates a backup of the linkding data folder"
+
+    def add_arguments(self, parser):
+        parser.add_argument("backup_file", type=str, help="Backup zip file destination")
+
+    def handle(self, *args, **options):
+        backup_file = options["backup_file"]
+        with zipfile.ZipFile(backup_file, "w", zipfile.ZIP_DEFLATED) as zip_file:
+            # Backup the database
+            self.stdout.write("Create database backup...")
+            with tempfile.TemporaryDirectory() as temp_dir:
+                backup_db_file = os.path.join(temp_dir, "db.sqlite3")
+                self.backup_database(backup_db_file)
+                zip_file.write(backup_db_file, "db.sqlite3")
+
+            # Backup the assets folder
+            if not os.path.exists(os.path.join("data", "assets")):
+                self.stdout.write(
+                    self.style.WARNING("No assets folder found. Skipping...")
+                )
+            else:
+                self.stdout.write("Backup bookmark assets...")
+                assets_folder = os.path.join("data", "assets")
+                for root, _, files in os.walk(assets_folder):
+                    for file in files:
+                        file_path = os.path.join(root, file)
+                        zip_file.write(file_path, os.path.join("assets", file))
+
+            # Backup the favicons folder
+            if not os.path.exists(os.path.join("data", "favicons")):
+                self.stdout.write(
+                    self.style.WARNING("No favicons folder found. Skipping...")
+                )
+            else:
+                self.stdout.write("Backup bookmark favicons...")
+                favicons_folder = os.path.join("data", "favicons")
+                for root, _, files in os.walk(favicons_folder):
+                    for file in files:
+                        file_path = os.path.join(root, file)
+                        zip_file.write(file_path, os.path.join("favicons", file))
+
+            # Backup the previews folder
+            if not os.path.exists(os.path.join("data", "previews")):
+                self.stdout.write(
+                    self.style.WARNING("No previews folder found. Skipping...")
+                )
+            else:
+                self.stdout.write("Backup bookmark previews...")
+                previews_folder = os.path.join("data", "previews")
+                for root, _, files in os.walk(previews_folder):
+                    for file in files:
+                        file_path = os.path.join(root, file)
+                        zip_file.write(file_path, os.path.join("previews", file))
+
+        self.stdout.write(self.style.SUCCESS(f"Backup created at {backup_file}"))
+
+    def backup_database(self, backup_db_file):
+        def progress(status, remaining, total):
+            self.stdout.write(f"Copied {total-remaining} of {total} pages...")
+
+        source_db = sqlite3.connect(os.path.join("data", "db.sqlite3"))
+        backup_db = sqlite3.connect(backup_db_file)
+        with backup_db:
+            source_db.backup(backup_db, pages=50, progress=progress)
+        backup_db.close()
+        source_db.close()

bookmarks/management/commands/migrate_tasks.py

@@ -0,0 +1,75 @@
+import json
+import os
+import sqlite3
+import importlib
+
+from django.core.management.base import BaseCommand
+
+
+class Command(BaseCommand):
+    help = "Migrate tasks from django-background-tasks to Huey"
+
+    def handle(self, *args, **options):
+        db = sqlite3.connect(os.path.join("data", "db.sqlite3"))
+
+        # Check if background_task table exists
+        cursor = db.cursor()
+        cursor.execute(
+            "SELECT name FROM sqlite_master WHERE type='table' AND name='background_task'"
+        )
+        row = cursor.fetchone()
+        if not row:
+            self.stdout.write(
+                "Legacy task table does not exist. Skipping task migration"
+            )
+            return
+
+        # Load legacy tasks
+        cursor.execute("SELECT id, task_name, task_params FROM background_task")
+        legacy_tasks = cursor.fetchall()
+
+        if len(legacy_tasks) == 0:
+            self.stdout.write("No legacy tasks found. Skipping task migration")
+            return
+
+        self.stdout.write(
+            f"Found {len(legacy_tasks)} legacy tasks. Migrating to Huey..."
+        )
+
+        # Migrate tasks to Huey
+        succeeded_tasks = []
+        for task in legacy_tasks:
+            task_id = task[0]
+            task_name = task[1]
+            task_params_json = task[2]
+            try:
+                task_params = json.loads(task_params_json)
+                function_params = task_params[0]
+
+                # Resolve task function
+                module_name, func_name = task_name.rsplit(".", 1)
+                module = importlib.import_module(module_name)
+                func = getattr(module, func_name)
+
+                # Call task function
+                func(*function_params)
+                succeeded_tasks.append(task_id)
+            except Exception:
+                self.stderr.write(f"Error migrating task [{task_id}] {task_name}")
+
+        self.stdout.write(f"Migrated {len(succeeded_tasks)} tasks successfully")
+
+        # Clean up
+        try:
+            placeholders = ", ".join("?" for _ in succeeded_tasks)
+            sql = f"DELETE FROM background_task WHERE id IN ({placeholders})"
+            cursor.execute(sql, succeeded_tasks)
+            db.commit()
+            self.stdout.write(
+                f"Deleted {len(succeeded_tasks)} migrated tasks from legacy table"
+            )
+        except Exception:
+            self.stderr.write("Error cleaning up legacy tasks")
+
+        cursor.close()
+        db.close()

bookmarks/middlewares.py

@@ -1,13 +1,17 @@
 from django.conf import settings
 from django.contrib.auth.middleware import RemoteUserMiddleware
 
-from bookmarks.models import UserProfile
+from bookmarks.models import UserProfile, GlobalSettings
 
 
 class CustomRemoteUserMiddleware(RemoteUserMiddleware):
     header = settings.LD_AUTH_PROXY_USERNAME_HEADER
 
 
+standard_profile = UserProfile()
+standard_profile.enable_favicons = True
+
+
 class UserProfileMiddleware:
     def __init__(self, get_response):
         self.get_response = get_response
@@ -16,8 +20,16 @@ class UserProfileMiddleware:
         if request.user.is_authenticated:
             request.user_profile = request.user.profile
         else:
-            request.user_profile = UserProfile()
+            # check if a custom profile for guests exists, otherwise use standard profile
-            request.user_profile.enable_favicons = True
+            guest_profile = None
+            try:
+                global_settings = GlobalSettings.get()
+                if global_settings.guest_profile_user:
+                    guest_profile = global_settings.guest_profile_user.profile
+            except:
+                pass
+
+            request.user_profile = guest_profile or standard_profile
 
         response = self.get_response(request)
 

bookmarks/migrations/0030_bookmarkasset.py

@@ -0,0 +1,43 @@
+# Generated by Django 5.0.2 on 2024-03-31 08:21
+
+import django.db.models.deletion
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ("bookmarks", "0029_bookmark_list_actions_toast"),
+    ]
+
+    operations = [
+        migrations.CreateModel(
+            name="BookmarkAsset",
+            fields=[
+                (
+                    "id",
+                    models.AutoField(
+                        auto_created=True,
+                        primary_key=True,
+                        serialize=False,
+                        verbose_name="ID",
+                    ),
+                ),
+                ("date_created", models.DateTimeField(auto_now_add=True)),
+                ("file", models.CharField(blank=True, max_length=2048)),
+                ("file_size", models.IntegerField(null=True)),
+                ("asset_type", models.CharField(max_length=64)),
+                ("content_type", models.CharField(max_length=128)),
+                ("display_name", models.CharField(blank=True, max_length=2048)),
+                ("status", models.CharField(max_length=64)),
+                ("gzip", models.BooleanField(default=False)),
+                (
+                    "bookmark",
+                    models.ForeignKey(
+                        on_delete=django.db.models.deletion.CASCADE,
+                        to="bookmarks.bookmark",
+                    ),
+                ),
+            ],
+        ),
+    ]

bookmarks/migrations/0031_userprofile_enable_automatic_html_snapshots.py

@@ -0,0 +1,18 @@
+# Generated by Django 5.0.2 on 2024-04-01 10:29
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ("bookmarks", "0030_bookmarkasset"),
+    ]
+
+    operations = [
+        migrations.AddField(
+            model_name="userprofile",
+            name="enable_automatic_html_snapshots",
+            field=models.BooleanField(default=True),
+        ),
+    ]

bookmarks/migrations/0032_html_snapshots_hint_toast.py

@@ -0,0 +1,34 @@
+# Generated by Django 5.0.2 on 2024-04-01 12:17
+
+from django.db import migrations
+from django.contrib.auth import get_user_model
+
+from bookmarks.models import Toast
+
+User = get_user_model()
+
+
+def forwards(apps, schema_editor):
+
+    for user in User.objects.all():
+        toast = Toast(
+            key="html_snapshots_hint",
+            message="This version adds a new feature for archiving snapshots of websites locally. To use it, you need to switch to a different Docker image. See the installation instructions on GitHub for details.",
+            owner=user,
+        )
+        toast.save()
+
+
+def reverse(apps, schema_editor):
+    Toast.objects.filter(key="bookmark_list_actions_hint").delete()
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ("bookmarks", "0031_userprofile_enable_automatic_html_snapshots"),
+    ]
+
+    operations = [
+        migrations.RunPython(forwards, reverse),
+    ]

bookmarks/migrations/0033_userprofile_default_mark_unread.py

@@ -0,0 +1,18 @@
+# Generated by Django 5.0.3 on 2024-04-17 19:27
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ("bookmarks", "0032_html_snapshots_hint_toast"),
+    ]
+
+    operations = [
+        migrations.AddField(
+            model_name="userprofile",
+            name="default_mark_unread",
+            field=models.BooleanField(default=False),
+        ),
+    ]

bookmarks/migrations/0034_bookmark_preview_image_file_and_more.py

@@ -0,0 +1,23 @@
+# Generated by Django 5.0.3 on 2024-05-10 07:01
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ("bookmarks", "0033_userprofile_default_mark_unread"),
+    ]
+
+    operations = [
+        migrations.AddField(
+            model_name="bookmark",
+            name="preview_image_file",
+            field=models.CharField(blank=True, max_length=512),
+        ),
+        migrations.AddField(
+            model_name="userprofile",
+            name="enable_preview_images",
+            field=models.BooleanField(default=False),
+        ),
+    ]

bookmarks/migrations/0035_userprofile_tag_grouping.py

@@ -0,0 +1,22 @@
+# Generated by Django 5.0.3 on 2024-05-14 08:28
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ("bookmarks", "0034_bookmark_preview_image_file_and_more"),
+    ]
+
+    operations = [
+        migrations.AddField(
+            model_name="userprofile",
+            name="tag_grouping",
+            field=models.CharField(
+                choices=[("alphabetical", "Alphabetical"), ("disabled", "Disabled")],
+                default="alphabetical",
+                max_length=12,
+            ),
+        ),
+    ]

bookmarks/migrations/0036_userprofile_auto_tagging_rules.py

@@ -0,0 +1,18 @@
+# Generated by Django 5.0.3 on 2024-05-17 07:09
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ("bookmarks", "0035_userprofile_tag_grouping"),
+    ]
+
+    operations = [
+        migrations.AddField(
+            model_name="userprofile",
+            name="auto_tagging_rules",
+            field=models.TextField(blank=True),
+        ),
+    ]

bookmarks/migrations/0037_globalsettings.py

@@ -0,0 +1,38 @@
+# Generated by Django 5.0.8 on 2024-08-31 12:39
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ("bookmarks", "0036_userprofile_auto_tagging_rules"),
+    ]
+
+    operations = [
+        migrations.CreateModel(
+            name="GlobalSettings",
+            fields=[
+                (
+                    "id",
+                    models.AutoField(
+                        auto_created=True,
+                        primary_key=True,
+                        serialize=False,
+                        verbose_name="ID",
+                    ),
+                ),
+                (
+                    "landing_page",
+                    models.CharField(
+                        choices=[
+                            ("login", "Login"),
+                            ("shared_bookmarks", "Shared Bookmarks"),
+                        ],
+                        default="login",
+                        max_length=50,
+                    ),
+                ),
+            ],
+        ),
+    ]

bookmarks/migrations/0038_globalsettings_guest_profile_user.py

@@ -0,0 +1,26 @@
+# Generated by Django 5.0.8 on 2024-08-31 17:54
+
+import django.db.models.deletion
+from django.conf import settings
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ("bookmarks", "0037_globalsettings"),
+        migrations.swappable_dependency(settings.AUTH_USER_MODEL),
+    ]
+
+    operations = [
+        migrations.AddField(
+            model_name="globalsettings",
+            name="guest_profile_user",
+            field=models.ForeignKey(
+                blank=True,
+                null=True,
+                on_delete=django.db.models.deletion.SET_NULL,
+                to=settings.AUTH_USER_MODEL,
+            ),
+        ),
+    ]

bookmarks/models.py

@@ -1,18 +1,22 @@
-import binascii
+import logging
 import os
 from typing import List
 
+import binascii
 from django import forms
+from django.conf import settings
 from django.contrib.auth import get_user_model
 from django.contrib.auth.models import User
 from django.db import models
-from django.db.models.signals import post_save
+from django.db.models.signals import post_save, post_delete
 from django.dispatch import receiver
 from django.http import QueryDict
 
 from bookmarks.utils import unique
 from bookmarks.validators import BookmarkURLValidator
 
+logger = logging.getLogger(__name__)
+
 
 class Tag(models.Model):
     name = models.CharField(max_length=64)
@@ -55,6 +59,7 @@ class Bookmark(models.Model):
     website_description = models.TextField(blank=True, null=True)
     web_archive_snapshot_url = models.CharField(max_length=2048, blank=True)
     favicon_file = models.CharField(max_length=512, blank=True)
+    preview_image_file = models.CharField(max_length=512, blank=True)
     unread = models.BooleanField(default=False)
     is_archived = models.BooleanField(default=False)
     shared = models.BooleanField(default=False)
@@ -79,12 +84,58 @@ class Bookmark(models.Model):
 
     @property
     def tag_names(self):
-        return [tag.name for tag in self.tags.all()]
+        names = [tag.name for tag in self.tags.all()]
+        return sorted(names)
 
     def __str__(self):
         return self.resolved_title + " (" + self.url[:30] + "...)"
 
 
+class BookmarkAsset(models.Model):
+    TYPE_SNAPSHOT = "snapshot"
+    TYPE_UPLOAD = "upload"
+
+    CONTENT_TYPE_HTML = "text/html"
+
+    STATUS_PENDING = "pending"
+    STATUS_COMPLETE = "complete"
+    STATUS_FAILURE = "failure"
+
+    bookmark = models.ForeignKey(Bookmark, on_delete=models.CASCADE)
+    date_created = models.DateTimeField(auto_now_add=True, null=False)
+    file = models.CharField(max_length=2048, blank=True, null=False)
+    file_size = models.IntegerField(null=True)
+    asset_type = models.CharField(max_length=64, blank=False, null=False)
+    content_type = models.CharField(max_length=128, blank=False, null=False)
+    display_name = models.CharField(max_length=2048, blank=True, null=False)
+    status = models.CharField(max_length=64, blank=False, null=False)
+    gzip = models.BooleanField(default=False, null=False)
+
+    def save(self, *args, **kwargs):
+        if self.file:
+            try:
+                file_path = os.path.join(settings.LD_ASSET_FOLDER, self.file)
+                if os.path.isfile(file_path):
+                    self.file_size = os.path.getsize(file_path)
+            except Exception:
+                pass
+        super().save(*args, **kwargs)
+
+    def __str__(self):
+        return self.display_name or f"Bookmark Asset #{self.pk}"
+
+
+@receiver(post_delete, sender=BookmarkAsset)
+def bookmark_asset_deleted(sender, instance, **kwargs):
+    if instance.file:
+        filepath = os.path.join(settings.LD_ASSET_FOLDER, instance.file)
+        if os.path.isfile(filepath):
+            try:
+                os.remove(filepath)
+            except Exception as error:
+                logger.error(f"Failed to delete asset file: {filepath}", exc_info=error)
+
+
 class BookmarkForm(forms.ModelForm):
     # Use URLField for URL
     url = forms.CharField(validators=[BookmarkURLValidator()])
@@ -119,7 +170,9 @@ class BookmarkForm(forms.ModelForm):
 
     @property
     def has_notes(self):
-        return self.instance and self.instance.notes
+        return self.initial.get("notes", None) or (
+            self.instance and self.instance.notes
+        )
 
 
 class BookmarkSearch:
@@ -302,6 +355,12 @@ class UserProfile(models.Model):
         (TAG_SEARCH_STRICT, "Strict"),
         (TAG_SEARCH_LAX, "Lax"),
     ]
+    TAG_GROUPING_ALPHABETICAL = "alphabetical"
+    TAG_GROUPING_DISABLED = "disabled"
+    TAG_GROUPING_CHOICES = [
+        (TAG_GROUPING_ALPHABETICAL, "Alphabetical"),
+        (TAG_GROUPING_DISABLED, "Disabled"),
+    ]
     user = models.OneToOneField(
         get_user_model(), related_name="profile", on_delete=models.CASCADE
     )
@@ -342,9 +401,16 @@ class UserProfile(models.Model):
         blank=False,
         default=TAG_SEARCH_STRICT,
     )
+    tag_grouping = models.CharField(
+        max_length=12,
+        choices=TAG_GROUPING_CHOICES,
+        blank=False,
+        default=TAG_GROUPING_ALPHABETICAL,
+    )
     enable_sharing = models.BooleanField(default=False, null=False)
     enable_public_sharing = models.BooleanField(default=False, null=False)
     enable_favicons = models.BooleanField(default=False, null=False)
+    enable_preview_images = models.BooleanField(default=False, null=False)
     display_url = models.BooleanField(default=False, null=False)
     display_view_bookmark_action = models.BooleanField(default=True, null=False)
     display_edit_bookmark_action = models.BooleanField(default=True, null=False)
@@ -352,7 +418,10 @@ class UserProfile(models.Model):
     display_remove_bookmark_action = models.BooleanField(default=True, null=False)
     permanent_notes = models.BooleanField(default=False, null=False)
     custom_css = models.TextField(blank=True, null=False)
+    auto_tagging_rules = models.TextField(blank=True, null=False)
     search_preferences = models.JSONField(default=dict, null=False)
+    enable_automatic_html_snapshots = models.BooleanField(default=True, null=False)
+    default_mark_unread = models.BooleanField(default=False, null=False)
 
 
 class UserProfileForm(forms.ModelForm):
@@ -366,16 +435,21 @@ class UserProfileForm(forms.ModelForm):
             "bookmark_link_target",
             "web_archive_integration",
             "tag_search",
+            "tag_grouping",
             "enable_sharing",
             "enable_public_sharing",
             "enable_favicons",
+            "enable_preview_images",
+            "enable_automatic_html_snapshots",
             "display_url",
             "display_view_bookmark_action",
             "display_edit_bookmark_action",
             "display_archive_bookmark_action",
             "display_remove_bookmark_action",
             "permanent_notes",
+            "default_mark_unread",
             "custom_css",
+            "auto_tagging_rules",
         ]
 
 
@@ -421,3 +495,45 @@ class FeedToken(models.Model):
 
     def __str__(self):
         return self.key
+
+
+class GlobalSettings(models.Model):
+    LANDING_PAGE_LOGIN = "login"
+    LANDING_PAGE_SHARED_BOOKMARKS = "shared_bookmarks"
+    LANDING_PAGE_CHOICES = [
+        (LANDING_PAGE_LOGIN, "Login"),
+        (LANDING_PAGE_SHARED_BOOKMARKS, "Shared Bookmarks"),
+    ]
+
+    landing_page = models.CharField(
+        max_length=50,
+        choices=LANDING_PAGE_CHOICES,
+        blank=False,
+        default=LANDING_PAGE_LOGIN,
+    )
+    guest_profile_user = models.ForeignKey(
+        get_user_model(), on_delete=models.SET_NULL, null=True, blank=True
+    )
+
+    @classmethod
+    def get(cls):
+        instance = GlobalSettings.objects.first()
+        if not instance:
+            instance = GlobalSettings()
+            instance.save()
+        return instance
+
+    def save(self, *args, **kwargs):
+        if not self.pk and GlobalSettings.objects.exists():
+            raise Exception("There is already one instance of GlobalSettings")
+        return super(GlobalSettings, self).save(*args, **kwargs)
+
+
+class GlobalSettingsForm(forms.ModelForm):
+    class Meta:
+        model = GlobalSettings
+        fields = ["landing_page", "guest_profile_user"]
+
+    def __init__(self, *args, **kwargs):
+        super(GlobalSettingsForm, self).__init__(*args, **kwargs)
+        self.fields["guest_profile_user"].empty_label = "Standard profile"

bookmarks/services/auto_tagging.py

@@ -0,0 +1,64 @@
+from urllib.parse import urlparse, parse_qs
+import re
+import idna
+
+
+def get_tags(script: str, url: str):
+    parsed_url = urlparse(url.lower())
+    result = set()
+
+    for line in script.lower().split("\n"):
+        if "#" in line:
+            i = line.index("#")
+            line = line[:i]
+
+        parts = line.split()
+        if len(parts) < 2:
+            continue
+
+        # to parse a host name from the pattern URL, ensure it has a scheme
+        pattern_url = "//" + re.sub("^https?://", "", parts[0])
+        parsed_pattern = urlparse(pattern_url)
+
+        if not _domains_matches(parsed_pattern.hostname, parsed_url.hostname):
+            continue
+
+        if parsed_pattern.path and not _path_matches(
+            parsed_pattern.path, parsed_url.path
+        ):
+            continue
+
+        if parsed_pattern.query and not _qs_matches(
+            parsed_pattern.query, parsed_url.query
+        ):
+            continue
+
+        for tag in parts[1:]:
+            result.add(tag)
+
+    return result
+
+
+def _path_matches(expected_path: str, actual_path: str) -> bool:
+    return actual_path.startswith(expected_path)
+
+
+def _domains_matches(expected_domain: str, actual_domain: str) -> bool:
+    expected_domain = idna.encode(expected_domain)
+    actual_domain = idna.encode(actual_domain)
+
+    return actual_domain.endswith(expected_domain)
+
+
+def _qs_matches(expected_qs: str, actual_qs: str) -> bool:
+    expected_qs = parse_qs(expected_qs, keep_blank_values=True)
+    actual_qs = parse_qs(actual_qs, keep_blank_values=True)
+
+    for key in expected_qs:
+        if key not in actual_qs:
+            return False
+        for value in expected_qs[key]:
+            if value != "" and value not in actual_qs[key]:
+                return False
+
+    return True

bookmarks/services/bookmarks.py

@@ -1,12 +1,19 @@
+import logging
+import os
 from typing import Union
 
+from django.conf import settings
 from django.contrib.auth.models import User
+from django.core.files.uploadedfile import UploadedFile
 from django.utils import timezone
 
-from bookmarks.models import Bookmark, parse_tag_string
+from bookmarks.models import Bookmark, BookmarkAsset, parse_tag_string
-from bookmarks.services.tags import get_or_create_tags
-from bookmarks.services import website_loader
 from bookmarks.services import tasks
+from bookmarks.services import website_loader
+from bookmarks.services import auto_tagging
+from bookmarks.services.tags import get_or_create_tags
+
+logger = logging.getLogger(__name__)
 
 
 def create_bookmark(bookmark: Bookmark, tag_string: str, current_user: User):
@@ -34,6 +41,11 @@ def create_bookmark(bookmark: Bookmark, tag_string: str, current_user: User):
     tasks.create_web_archive_snapshot(current_user, bookmark, False)
     # Load favicon
     tasks.load_favicon(current_user, bookmark)
+    # Load preview image
+    tasks.load_preview_image(current_user, bookmark)
+    # Create HTML snapshot
+    if current_user.profile.enable_automatic_html_snapshots:
+        tasks.create_html_snapshot(bookmark)
 
     return bookmark
 
@@ -49,6 +61,8 @@ def update_bookmark(bookmark: Bookmark, tag_string, current_user: User):
     bookmark.save()
     # Update favicon
     tasks.load_favicon(current_user, bookmark)
+    # Update preview image
+    tasks.load_preview_image(current_user, bookmark)
 
     if has_url_changed:
         # Update web archive snapshot, if URL changed
@@ -173,6 +187,46 @@ def unshare_bookmarks(bookmark_ids: [Union[int, str]], current_user: User):
     )
 
 
+def _generate_upload_asset_filename(asset: BookmarkAsset, filename: str):
+    formatted_datetime = asset.date_created.strftime("%Y-%m-%d_%H%M%S")
+    return f"{asset.asset_type}_{formatted_datetime}_{filename}"
+
+
+def upload_asset(bookmark: Bookmark, upload_file: UploadedFile) -> BookmarkAsset:
+    asset = BookmarkAsset(
+        bookmark=bookmark,
+        asset_type=BookmarkAsset.TYPE_UPLOAD,
+        content_type=upload_file.content_type,
+        display_name=upload_file.name,
+        status=BookmarkAsset.STATUS_PENDING,
+        gzip=False,
+    )
+    asset.save()
+
+    try:
+        filename = _generate_upload_asset_filename(asset, upload_file.name)
+        filepath = os.path.join(settings.LD_ASSET_FOLDER, filename)
+        with open(filepath, "wb") as f:
+            for chunk in upload_file.chunks():
+                f.write(chunk)
+        asset.status = BookmarkAsset.STATUS_COMPLETE
+        asset.file = filename
+        asset.file_size = upload_file.size
+        logger.info(
+            f"Successfully uploaded asset file. bookmark={bookmark} file={upload_file.name}"
+        )
+    except Exception as e:
+        logger.error(
+            f"Failed to upload asset file. bookmark={bookmark} file={upload_file.name}",
+            exc_info=e,
+        )
+        asset.status = BookmarkAsset.STATUS_FAILURE
+
+    asset.save()
+
+    return asset
+
+
 def _merge_bookmark_data(from_bookmark: Bookmark, to_bookmark: Bookmark):
     to_bookmark.title = from_bookmark.title
     to_bookmark.description = from_bookmark.description
@@ -189,6 +243,21 @@ def _update_website_metadata(bookmark: Bookmark):
 
 def _update_bookmark_tags(bookmark: Bookmark, tag_string: str, user: User):
     tag_names = parse_tag_string(tag_string)
+
+    if user.profile.auto_tagging_rules:
+        try:
+            auto_tag_names = auto_tagging.get_tags(
+                user.profile.auto_tagging_rules, bookmark.url
+            )
+            for auto_tag_name in auto_tag_names:
+                if auto_tag_name not in tag_names:
+                    tag_names.append(auto_tag_name)
+        except Exception as e:
+            logger.error(
+                f"Failed to auto-tag bookmark. url={bookmark.url}",
+                exc_info=e,
+            )
+
     tags = get_or_create_tags(tag_names, user)
     bookmark.tags.set(tags)
 

bookmarks/services/importer.py

@@ -79,10 +79,10 @@ def import_netscape_html(
     for batch in batches:
         _import_batch(batch, user, options, tag_cache, result)
 
-    # Create snapshots for newly imported bookmarks
-    tasks.schedule_bookmarks_without_snapshots(user)
     # Load favicons for newly imported bookmarks
     tasks.schedule_bookmarks_without_favicons(user)
+    # Load previews for newly imported bookmarks
+    tasks.schedule_bookmarks_without_previews(user)
 
     end = timezone.now()
     logger.debug(f"Import duration: {end - import_start}")

bookmarks/services/monolith.py

@@ -0,0 +1,32 @@
+import gzip
+import shutil
+import subprocess
+import os
+
+from django.conf import settings
+
+
+class MonolithError(Exception):
+    pass
+
+
+# Monolith isn't used at the moment, as the local snapshot implementation
+# switched to single-file after the prototype. Keeping this around in case
+# it turns out to be useful in the future.
+def create_snapshot(url: str, filepath: str):
+    monolith_path = settings.LD_MONOLITH_PATH
+    monolith_options = settings.LD_MONOLITH_OPTIONS
+    temp_filepath = filepath + ".tmp"
+
+    try:
+        command = f"{monolith_path} '{url}' {monolith_options} -o {temp_filepath}"
+        subprocess.run(command, check=True, shell=True)
+
+        with open(temp_filepath, "rb") as raw_file, gzip.open(
+            filepath, "wb"
+        ) as gz_file:
+            shutil.copyfileobj(raw_file, gz_file)
+
+        os.remove(temp_filepath)
+    except subprocess.CalledProcessError as error:
+        raise MonolithError(f"Failed to create snapshot: {error.stderr}")

bookmarks/services/preview_image_loader.py

@@ -0,0 +1,88 @@
+import logging
+import mimetypes
+import os.path
+import hashlib
+from pathlib import Path
+
+import requests
+from django.conf import settings
+from bookmarks.services import website_loader
+
+logger = logging.getLogger(__name__)
+
+
+def _ensure_preview_folder():
+    Path(settings.LD_PREVIEW_FOLDER).mkdir(parents=True, exist_ok=True)
+
+
+def _url_to_filename(preview_image: str) -> str:
+    return hashlib.md5(preview_image.encode()).hexdigest()
+
+
+def _get_image_path(preview_image_file: str) -> Path:
+    return Path(os.path.join(settings.LD_PREVIEW_FOLDER, preview_image_file))
+
+
+def load_preview_image(url: str) -> str | None:
+    _ensure_preview_folder()
+
+    metadata = website_loader.load_website_metadata(url)
+    if not metadata.preview_image:
+        logger.debug(f"Could not find preview image in metadata: {url}")
+        return None
+
+    image_url = metadata.preview_image
+
+    logger.debug(f"Loading preview image: {image_url}")
+    with requests.get(image_url, stream=True) as response:
+        if response.status_code < 200 or response.status_code >= 300:
+            logger.debug(
+                f"Bad response status code for preview image: {image_url} status_code={response.status_code}"
+            )
+            return None
+
+        if "Content-Length" not in response.headers:
+            logger.debug(f"Empty Content-Length for preview image: {image_url}")
+            return None
+
+        content_length = int(response.headers["Content-Length"])
+        if content_length > settings.LD_PREVIEW_MAX_SIZE:
+            logger.debug(
+                f"Content-Length exceeds LD_PREVIEW_MAX_SIZE: {image_url} length={content_length}"
+            )
+            return None
+
+        if "Content-Type" not in response.headers:
+            logger.debug(f"Empty Content-Type for preview image: {image_url}")
+            return None
+
+        content_type = response.headers["Content-Type"].split(";", 1)[0]
+        file_extension = mimetypes.guess_extension(content_type)
+
+        if file_extension not in settings.LD_PREVIEW_ALLOWED_EXTENSIONS:
+            logger.debug(
+                f"Unsupported Content-Type for preview image: {image_url} content_type={content_type}"
+            )
+            return None
+
+        preview_image_hash = _url_to_filename(url)
+        preview_image_file = f"{preview_image_hash}{file_extension}"
+        preview_image_path = _get_image_path(preview_image_file)
+
+        with open(preview_image_path, "wb") as file:
+            downloaded = 0
+            for chunk in response.iter_content(chunk_size=8192):
+                downloaded += len(chunk)
+                if downloaded > content_length:
+                    logger.debug(
+                        f"Content-Length mismatch for preview image: {image_url} length={content_length} downloaded={downloaded}"
+                    )
+                    file.close()
+                    preview_image_path.unlink()
+                    return None
+
+                file.write(chunk)
+
+    logger.debug(f"Saved preview image as: {preview_image_path}")
+
+    return preview_image_file

bookmarks/services/singlefile.py

@@ -0,0 +1,58 @@
+import gzip
+import logging
+import os
+import shlex
+import shutil
+import signal
+import subprocess
+
+from django.conf import settings
+
+
+class SingeFileError(Exception):
+    pass
+
+
+logger = logging.getLogger(__name__)
+
+
+def create_snapshot(url: str, filepath: str):
+    singlefile_path = settings.LD_SINGLEFILE_PATH
+    # parse options to list of arguments
+    ublock_options = shlex.split(settings.LD_SINGLEFILE_UBLOCK_OPTIONS)
+    custom_options = shlex.split(settings.LD_SINGLEFILE_OPTIONS)
+    temp_filepath = filepath + ".tmp"
+    # concat lists
+    args = [singlefile_path] + ublock_options + custom_options + [url, temp_filepath]
+    try:
+        # Use start_new_session=True to create a new process group
+        process = subprocess.Popen(args, start_new_session=True)
+        process.wait(timeout=settings.LD_SINGLEFILE_TIMEOUT_SEC)
+
+        # check if the file was created
+        if not os.path.exists(temp_filepath):
+            raise SingeFileError("Failed to create snapshot")
+
+        with open(temp_filepath, "rb") as raw_file, gzip.open(
+            filepath, "wb"
+        ) as gz_file:
+            shutil.copyfileobj(raw_file, gz_file)
+
+        os.remove(temp_filepath)
+    except subprocess.TimeoutExpired:
+        # First try to terminate properly
+        try:
+            logger.error(
+                "Timeout expired while creating snapshot. Terminating process..."
+            )
+            process.terminate()
+            process.wait(timeout=20)
+            raise SingeFileError("Timeout expired while creating snapshot")
+        except subprocess.TimeoutExpired:
+            # Kill the whole process group, which should also clean up any chromium
+            # processes spawned by single-file
+            logger.error("Timeout expired while terminating. Killing process...")
+            os.killpg(os.getpgid(process.pid), signal.SIGTERM)
+            raise SingeFileError("Timeout expired while creating snapshot")
+    except subprocess.CalledProcessError as error:
+        raise SingeFileError(f"Failed to create snapshot: {error.stderr}")

bookmarks/services/tasks.py

@@ -1,21 +1,55 @@
+import functools
 import logging
+import os
+from typing import List
 
 import waybackpy
-from background_task import background
-from background_task.models import Task
 from django.conf import settings
 from django.contrib.auth import get_user_model
 from django.contrib.auth.models import User
-from waybackpy.exceptions import WaybackError, TooManyRequestsError, NoCDXRecordFound
+from django.db.models import Q
+from django.utils import timezone, formats
+from huey import crontab
+from huey.contrib.djhuey import HUEY as huey
+from huey.exceptions import TaskLockedException
+from waybackpy.exceptions import WaybackError, TooManyRequestsError
 
-import bookmarks.services.wayback
+from bookmarks.models import Bookmark, BookmarkAsset, UserProfile
-from bookmarks.models import Bookmark, UserProfile
+from bookmarks.services import favicon_loader, singlefile, preview_image_loader
-from bookmarks.services import favicon_loader
 from bookmarks.services.website_loader import DEFAULT_USER_AGENT
 
 logger = logging.getLogger(__name__)
 
 
+# Create custom decorator for Huey tasks that implements exponential backoff
+# Taken from: https://huey.readthedocs.io/en/latest/guide.html#tips-and-tricks
+# Retry 1: 60
+# Retry 2: 240
+# Retry 3: 960
+# Retry 4: 3840
+# Retry 5: 15360
+def task(retries=5, retry_delay=15, retry_backoff=4):
+    def deco(fn):
+        @functools.wraps(fn)
+        def inner(*args, **kwargs):
+            task = kwargs.pop("task")
+            try:
+                return fn(*args, **kwargs)
+            except TaskLockedException as exc:
+                # Task locks are currently only used as workaround to enforce
+                # running specific types of tasks (e.g. singlefile snapshots)
+                # sequentially. In that case don't reduce the number of retries.
+                task.retries = retries
+                raise exc
+            except Exception as exc:
+                task.retry_delay *= retry_backoff
+                raise exc
+
+        return huey.task(retries=retries, retry_delay=retry_delay, context=True)(inner)
+
+    return deco
+
+
 def is_web_archive_integration_active(user: User) -> bool:
     background_tasks_enabled = not settings.LD_DISABLE_BACKGROUND_TASKS
     web_archive_integration_enabled = (
@@ -31,29 +65,6 @@ def create_web_archive_snapshot(user: User, bookmark: Bookmark, force_update: bo
         _create_web_archive_snapshot_task(bookmark.id, force_update)
 
 
-def _load_newest_snapshot(bookmark: Bookmark):
-    try:
-        logger.info(f"Load existing snapshot for bookmark. url={bookmark.url}")
-        cdx_api = bookmarks.services.wayback.CustomWaybackMachineCDXServerAPI(
-            bookmark.url
-        )
-        existing_snapshot = cdx_api.newest()
-
-        if existing_snapshot:
-            bookmark.web_archive_snapshot_url = existing_snapshot.archive_url
-            bookmark.save(update_fields=["web_archive_snapshot_url"])
-            logger.info(
-                f"Using newest snapshot. url={bookmark.url} from={existing_snapshot.datetime_timestamp}"
-            )
-
-    except NoCDXRecordFound:
-        logger.info(f"Could not find any snapshots for bookmark. url={bookmark.url}")
-    except WaybackError as error:
-        logger.error(
-            f"Failed to load existing snapshot. url={bookmark.url}", exc_info=error
-        )
-
-
 def _create_snapshot(bookmark: Bookmark):
     logger.info(f"Create new snapshot for bookmark. url={bookmark.url}...")
     archive = waybackpy.WaybackMachineSaveAPI(
@@ -65,7 +76,7 @@ def _create_snapshot(bookmark: Bookmark):
     logger.info(f"Successfully created new snapshot for bookmark:. url={bookmark.url}")
 
 
-@background()
+@task()
 def _create_web_archive_snapshot_task(bookmark_id: int, force_update: bool):
     try:
         bookmark = Bookmark.objects.get(id=bookmark_id)
@@ -82,47 +93,27 @@ def _create_web_archive_snapshot_task(bookmark_id: int, force_update: bool):
         return
     except TooManyRequestsError:
         logger.error(
-            f"Failed to create snapshot due to rate limiting, trying to load newest snapshot as fallback. url={bookmark.url}"
+            f"Failed to create snapshot due to rate limiting. url={bookmark.url}"
         )
     except WaybackError as error:
         logger.error(
-            f"Failed to create snapshot, trying to load newest snapshot as fallback. url={bookmark.url}",
+            f"Failed to create snapshot. url={bookmark.url}",
             exc_info=error,
         )
 
-    # Load the newest snapshot as fallback
-    _load_newest_snapshot(bookmark)
 
-
+@task()
-@background()
 def _load_web_archive_snapshot_task(bookmark_id: int):
-    try:
+    # Loading snapshots from CDX API has been removed, keeping the task function
-        bookmark = Bookmark.objects.get(id=bookmark_id)
+    # for now to prevent errors when huey tries to run the task
-    except Bookmark.DoesNotExist:
+    pass
-        return
-    # Skip if snapshot exists
-    if bookmark.web_archive_snapshot_url:
-        return
-    # Load the newest snapshot
-    _load_newest_snapshot(bookmark)
 
 
-def schedule_bookmarks_without_snapshots(user: User):
+@task()
-    if is_web_archive_integration_active(user):
-        _schedule_bookmarks_without_snapshots_task(user.id)
-
-
-@background()
 def _schedule_bookmarks_without_snapshots_task(user_id: int):
-    user = get_user_model().objects.get(id=user_id)
+    # Loading snapshots from CDX API has been removed, keeping the task function
-    bookmarks_without_snapshots = Bookmark.objects.filter(
+    # for now to prevent errors when huey tries to run the task
-        web_archive_snapshot_url__exact="", owner=user
+    pass
-    )
-
-    for bookmark in bookmarks_without_snapshots:
-        # To prevent rate limit errors from the Wayback API only try to load the latest snapshots instead of creating
-        # new ones when processing bookmarks in bulk
-        _load_web_archive_snapshot_task(bookmark.id)
 
 
 def is_favicon_feature_active(user: User) -> bool:
@@ -131,12 +122,18 @@ def is_favicon_feature_active(user: User) -> bool:
     return background_tasks_enabled and user.profile.enable_favicons
 
 
+def is_preview_feature_active(user: User) -> bool:
+    return (
+        user.profile.enable_preview_images and not settings.LD_DISABLE_BACKGROUND_TASKS
+    )
+
+
 def load_favicon(user: User, bookmark: Bookmark):
     if is_favicon_feature_active(user):
         _load_favicon_task(bookmark.id)
 
 
-@background()
+@task()
 def _load_favicon_task(bookmark_id: int):
     try:
         bookmark = Bookmark.objects.get(id=bookmark_id)
@@ -160,19 +157,15 @@ def schedule_bookmarks_without_favicons(user: User):
         _schedule_bookmarks_without_favicons_task(user.id)
 
 
-@background()
+@task()
 def _schedule_bookmarks_without_favicons_task(user_id: int):
     user = get_user_model().objects.get(id=user_id)
     bookmarks = Bookmark.objects.filter(favicon_file__exact="", owner=user)
-    tasks = []
 
+    # TODO: Implement bulk task creation
     for bookmark in bookmarks:
-        task = Task.objects.new_task(
+        _load_favicon_task(bookmark.id)
-            task_name="bookmarks.services.tasks._load_favicon_task", args=(bookmark.id,)
+        pass
-        )
-        tasks.append(task)
-
-    Task.objects.bulk_create(tasks)
 
 
 def schedule_refresh_favicons(user: User):
@@ -180,16 +173,180 @@ def schedule_refresh_favicons(user: User):
         _schedule_refresh_favicons_task(user.id)
 
 
-@background()
+@task()
 def _schedule_refresh_favicons_task(user_id: int):
     user = get_user_model().objects.get(id=user_id)
     bookmarks = Bookmark.objects.filter(owner=user)
-    tasks = []
 
+    # TODO: Implement bulk task creation
     for bookmark in bookmarks:
-        task = Task.objects.new_task(
+        _load_favicon_task(bookmark.id)
-            task_name="bookmarks.services.tasks._load_favicon_task", args=(bookmark.id,)
-        )
-        tasks.append(task)
 
-    Task.objects.bulk_create(tasks)
+
+def load_preview_image(user: User, bookmark: Bookmark):
+    if is_preview_feature_active(user):
+        _load_preview_image_task(bookmark.id)
+
+
+@task()
+def _load_preview_image_task(bookmark_id: int):
+    try:
+        bookmark = Bookmark.objects.get(id=bookmark_id)
+    except Bookmark.DoesNotExist:
+        return
+
+    logger.info(f"Load preview image for bookmark. url={bookmark.url}")
+
+    new_preview_image_file = preview_image_loader.load_preview_image(bookmark.url)
+
+    if new_preview_image_file != bookmark.preview_image_file:
+        bookmark.preview_image_file = new_preview_image_file or ""
+        bookmark.save(update_fields=["preview_image_file"])
+        logger.info(
+            f"Successfully updated preview image for bookmark. url={bookmark.url} preview_image_file={new_preview_image_file}"
+        )
+
+
+def schedule_bookmarks_without_previews(user: User):
+    if is_preview_feature_active(user):
+        _schedule_bookmarks_without_previews_task(user.id)
+
+
+@task()
+def _schedule_bookmarks_without_previews_task(user_id: int):
+    user = get_user_model().objects.get(id=user_id)
+    bookmarks = Bookmark.objects.filter(
+        Q(preview_image_file__exact=""),
+        owner=user,
+    )
+
+    # TODO: Implement bulk task creation
+    for bookmark in bookmarks:
+        try:
+            _load_preview_image_task(bookmark.id)
+        except Exception as exc:
+            logging.exception(exc)
+
+
+def is_html_snapshot_feature_active() -> bool:
+    return settings.LD_ENABLE_SNAPSHOTS and not settings.LD_DISABLE_BACKGROUND_TASKS
+
+
+def create_html_snapshot(bookmark: Bookmark):
+    if not is_html_snapshot_feature_active():
+        return
+
+    asset = _create_snapshot_asset(bookmark)
+    asset.save()
+
+
+def create_html_snapshots(bookmark_list: List[Bookmark]):
+    if not is_html_snapshot_feature_active():
+        return
+
+    assets_to_create = []
+    for bookmark in bookmark_list:
+        asset = _create_snapshot_asset(bookmark)
+        assets_to_create.append(asset)
+
+    BookmarkAsset.objects.bulk_create(assets_to_create)
+
+
+MAX_SNAPSHOT_FILENAME_LENGTH = 192
+
+
+def _create_snapshot_asset(bookmark: Bookmark) -> BookmarkAsset:
+    timestamp = formats.date_format(timezone.now(), "SHORT_DATE_FORMAT")
+    asset = BookmarkAsset(
+        bookmark=bookmark,
+        asset_type=BookmarkAsset.TYPE_SNAPSHOT,
+        content_type="text/html",
+        display_name=f"HTML snapshot from {timestamp}",
+        status=BookmarkAsset.STATUS_PENDING,
+    )
+    return asset
+
+
+def _generate_snapshot_filename(asset: BookmarkAsset) -> str:
+    def sanitize_char(char):
+        if char.isalnum() or char in ("-", "_", "."):
+            return char
+        else:
+            return "_"
+
+    formatted_datetime = asset.date_created.strftime("%Y-%m-%d_%H%M%S")
+    sanitized_url = "".join(sanitize_char(char) for char in asset.bookmark.url)
+
+    # Calculate the length of the non-URL parts of the filename
+    non_url_length = len(f"{asset.asset_type}{formatted_datetime}__.html.gz")
+    # Calculate the maximum length for the URL part
+    max_url_length = MAX_SNAPSHOT_FILENAME_LENGTH - non_url_length
+    # Truncate the URL if necessary
+    sanitized_url = sanitized_url[:max_url_length]
+
+    return f"{asset.asset_type}_{formatted_datetime}_{sanitized_url}.html.gz"
+
+
+# singe-file does not support running multiple instances in parallel, so we can
+# not queue up multiple snapshot tasks at once. Instead, schedule a periodic
+# task that grabs a number of pending assets and creates snapshots for them in
+# sequence. The task uses a lock to ensure that a new task isn't scheduled
+# before the previous one has finished.
+@huey.periodic_task(crontab(minute="*"))
+@huey.lock_task("schedule-html-snapshots-lock")
+def _schedule_html_snapshots_task():
+    # Get five pending assets
+    assets = BookmarkAsset.objects.filter(status=BookmarkAsset.STATUS_PENDING).order_by(
+        "date_created"
+    )[:5]
+
+    for asset in assets:
+        _create_html_snapshot_task(asset.id)
+
+
+def _create_html_snapshot_task(asset_id: int):
+    try:
+        asset = BookmarkAsset.objects.get(id=asset_id)
+    except BookmarkAsset.DoesNotExist:
+        return
+
+    logger.info(f"Create HTML snapshot for bookmark. url={asset.bookmark.url}")
+
+    try:
+        filename = _generate_snapshot_filename(asset)
+        filepath = os.path.join(settings.LD_ASSET_FOLDER, filename)
+        singlefile.create_snapshot(asset.bookmark.url, filepath)
+        asset.status = BookmarkAsset.STATUS_COMPLETE
+        asset.file = filename
+        asset.gzip = True
+        asset.save()
+        logger.info(
+            f"Successfully created HTML snapshot for bookmark. url={asset.bookmark.url}"
+        )
+    except Exception as error:
+        logger.error(
+            f"Failed to HTML snapshot for bookmark. url={asset.bookmark.url}",
+            exc_info=error,
+        )
+        asset.status = BookmarkAsset.STATUS_FAILURE
+        asset.save()
+
+
+def create_missing_html_snapshots(user: User) -> int:
+    if not is_html_snapshot_feature_active():
+        return 0
+
+    bookmarks_without_snapshots = Bookmark.objects.filter(owner=user).exclude(
+        bookmarkasset__asset_type=BookmarkAsset.TYPE_SNAPSHOT,
+        bookmarkasset__status__in=[
+            BookmarkAsset.STATUS_PENDING,
+            BookmarkAsset.STATUS_COMPLETE,
+        ],
+    )
+    bookmarks_without_snapshots |= Bookmark.objects.filter(owner=user).exclude(
+        bookmarkasset__asset_type=BookmarkAsset.TYPE_SNAPSHOT
+    )
+
+    create_html_snapshots(list(bookmarks_without_snapshots))
+
+    return bookmarks_without_snapshots.count()

bookmarks/services/wayback.py

@@ -1,42 +1,20 @@
-import time
+import datetime
-from typing import Dict
 
-import waybackpy
+from django.utils import timezone
-import waybackpy.utils
-from waybackpy.exceptions import NoCDXRecordFound
 
 
-class CustomWaybackMachineCDXServerAPI(waybackpy.WaybackMachineCDXServerAPI):
+def generate_fallback_webarchive_url(
+    url: str, timestamp: datetime.datetime
+) -> str | None:
     """
-    Customized WaybackMachineCDXServerAPI to work around some issues with retrieving the newest snapshot.
+    Generate a URL to the web archive for the given URL and timestamp.
-    See https://github.com/akamhy/waybackpy/issues/176
+    A snapshot for the specific timestamp might not exist, in which case the
+    web archive will show the closest snapshot to the given timestamp.
+    If there is no snapshot at all the URL will be invalid.
     """
+    if not url:
+        return None
+    if not timestamp:
+        timestamp = timezone.now()
 
-    def newest(self):
+    return f"https://web.archive.org/web/{timestamp.strftime('%Y%m%d%H%M%S')}/{url}"
-        unix_timestamp = int(time.time())
-        self.closest = waybackpy.utils.unix_timestamp_to_wayback_timestamp(
-            unix_timestamp
-        )
-        self.sort = "closest"
-        self.limit = -5
-
-        newest_snapshot = None
-        for snapshot in self.snapshots():
-            newest_snapshot = snapshot
-            break
-
-        if not newest_snapshot:
-            raise NoCDXRecordFound(
-                "Wayback Machine's CDX server did not return any records "
-                + "for the query. The URL may not have any archives "
-                + " on the Wayback Machine or the URL may have been recently "
-                + "archived and is still not available on the CDX server."
-            )
-
-        return newest_snapshot
-
-    def add_payload(self, payload: Dict[str, str]) -> None:
-        super().add_payload(payload)
-        # Set fastLatest query param, as we are only using this API to get the latest snapshot and using fastLatest
-        # makes searching for latest snapshots faster
-        payload["fastLatest"] = "true"

bookmarks/services/website_loader.py

@@ -1,6 +1,7 @@
 import logging
 from dataclasses import dataclass
 from functools import lru_cache
+from urllib.parse import urljoin
 
 import requests
 from bs4 import BeautifulSoup
@@ -15,12 +16,14 @@ class WebsiteMetadata:
     url: str
     title: str
     description: str
+    preview_image: str | None
 
     def to_dict(self):
         return {
             "url": self.url,
             "title": self.title,
             "description": self.description,
+            "preview_image": self.preview_image,
         }
 
 
@@ -30,6 +33,7 @@ class WebsiteMetadata:
 def load_website_metadata(url: str):
     title = None
     description = None
+    preview_image = None
     try:
         start = timezone.now()
         page_text = load_page(url)
@@ -55,10 +59,21 @@ def load_website_metadata(url: str):
                 else None
             )
 
+        image_tag = soup.find("meta", attrs={"property": "og:image"})
+        preview_image = image_tag["content"].strip() if image_tag else None
+        if (
+            preview_image
+            and not preview_image.startswith("http://")
+            and not preview_image.startswith("https://")
+        ):
+            preview_image = urljoin(url, preview_image)
+
         end = timezone.now()
         logger.debug(f"Parsing duration: {end - start}")
     finally:
-        return WebsiteMetadata(url=url, title=title, description=description)
+        return WebsiteMetadata(
+            url=url, title=title, description=description, preview_image=preview_image
+        )
 
 
 CHUNK_SIZE = 50 * 1024

bookmarks/signals.py

@@ -1,15 +1,7 @@
 from django.conf import settings
-from django.contrib.auth import user_logged_in
 from django.db.backends.signals import connection_created
 from django.dispatch import receiver
 
-from bookmarks.services import tasks
-
-
-@receiver(user_logged_in)
-def user_logged_in(sender, request, user, **kwargs):
-    tasks.schedule_bookmarks_without_snapshots(user)
-
 
 @receiver(connection_created)
 def extend_sqlite(connection=None, **kwargs):

bookmarks/styles/bookmark-details.scss

@@ -33,10 +33,72 @@
     text-overflow: ellipsis;
   }
 
+  .preview-image {
+    margin: $unit-4 0;
+
+    img {
+      max-width: 100%;
+      max-height: 200px;
+    }
+  }
+
   dl {
     margin-bottom: 0;
   }
 
+  .assets {
+    margin-top: $unit-2;
+  }
+
+  .assets .asset {
+    display: flex;
+    align-items: center;
+    gap: $unit-3;
+    padding: $unit-2 0;
+    border-top: $unit-o solid $border-color-light;
+  }
+
+  .assets .asset:last-child {
+    border-bottom: $unit-o solid $border-color-light;
+  }
+
+  .assets .asset-icon {
+    display: flex;
+    align-items: center;
+    justify-content: center;
+  }
+
+  .assets .asset-text {
+    flex: 1 1 0;
+    gap: $unit-2;
+    min-width: 0;
+    display: flex;
+  }
+
+  .assets .asset-text .truncate {
+    flex-shrink: 1;
+  }
+
+  .assets .asset-text .filesize {
+    color: $gray-color;
+  }
+
+  .assets .asset-actions, .assets-actions {
+    display: flex;
+    gap: $unit-4;
+    align-items: center;
+  }
+
+  .assets .asset-actions .btn, .assets-actions .btn {
+    height: unset;
+    padding: 0;
+    border: none;
+  }
+
+  .assets-actions {
+    margin-top: $unit-2;
+  }
+
   .tags a {
     color: $alternative-color;
   }
@@ -46,7 +108,7 @@
     gap: $unit-2;
   }
 
-  .status form .form-group, .status form .form-switch {
+  .status .form-group, .status .form-switch {
     margin: 0;
   }
 

bookmarks/styles/bookmark-form.scss

@@ -36,12 +36,11 @@
   .form-input-hint.bookmark-exists {
     display: none;
     color: $warning-color;
+  }
 
-    a {
+  .form-input-hint.auto-tags {
-      color: $warning-color;
+    display: none;
-      text-decoration: underline;
+    color: $success-color;
-      font-weight: bold;
-    }
   }
 
   details.notes textarea {

bookmarks/styles/bookmark-page.scss

@@ -128,9 +128,26 @@ ul.bookmark-list {
 /* Bookmarks */
 li[ld-bookmark-item] {
   position: relative;
+  display: flex;
+  gap: $unit-2;
   margin-top: $unit-2;
 
-  [ld-bulk-edit-checkbox].form-checkbox {
+  .content {
+    flex: 1 1 0;
+    min-width: 0;
+  }
+
+  img.preview-image {
+    flex: 0 0 auto;
+    width: 100px;
+    height: 60px;
+    margin-top: $unit-h;
+    object-fit: cover;
+    border-radius: $border-radius;
+    border: solid 1px $border-color-dark;
+  }
+
+  .form-checkbox.bulk-edit-checkbox {
     display: none;
   }
 
@@ -182,6 +199,12 @@ li[ld-bookmark-item] {
     animation: 0.3s ease 0s appear;
   }
 
+  @media (pointer:coarse) {
+    .title a[data-tooltip]::after {
+      display: none;
+    }
+  }
+
   &.unread .title a {
     font-style: italic;
   }
@@ -323,7 +346,7 @@ $bulk-edit-transition-duration: 400ms;
   }
 
   /* All checkbox */
-  [ld-bulk-edit-checkbox][all].form-checkbox {
+  .form-checkbox.bulk-edit-checkbox.all {
     display: block;
     width: $bulk-edit-toggle-width;
     margin: 0 0 0 $bulk-edit-toggle-offset;
@@ -331,7 +354,7 @@ $bulk-edit-transition-duration: 400ms;
   }
 
   /* Bookmark checkboxes */
-  li[ld-bookmark-item] [ld-bulk-edit-checkbox].form-checkbox {
+  li[ld-bookmark-item] .form-checkbox.bulk-edit-checkbox {
     display: block;
     position: absolute;
     width: $bulk-edit-toggle-width;
@@ -346,11 +369,11 @@ $bulk-edit-transition-duration: 400ms;
     transition: all $bulk-edit-transition-duration;
 
     .form-icon {
-        top: 0;
+      top: 0;
     }
   }
 
-  &.active li[ld-bookmark-item] [ld-bulk-edit-checkbox].form-checkbox {
+  &.active li[ld-bookmark-item] .form-checkbox.bulk-edit-checkbox {
     visibility: visible;
     opacity: 1;
   }

bookmarks/styles/reader-mode.scss

@@ -0,0 +1,27 @@
+html.reader-mode {
+  --font-size: 1rem;
+  line-height: 1.6;
+
+  body {
+    margin: 3rem 2rem;
+  }
+
+  .container {
+    max-width: 600px;
+  }
+
+  .byline {
+    font-style: italic;
+    font-size: 0.8rem;
+  }
+
+  .reading-time {
+    font-size: 0.7rem;
+  }
+
+  img {
+    max-width: 100%;
+    height: auto;
+  }
+}
+

bookmarks/styles/settings.scss

@@ -7,8 +7,9 @@
     }
   }
 
-  textarea.custom-css {
+  textarea.monospace {
     font-family: monospace;
+    box-sizing: border-box;
   }
 
   .input-group > input[type=submit] {

bookmarks/styles/spectre.scss

@@ -195,3 +195,10 @@ ul.menu li:first-child {
     font-size: 16px;
   }
 }
+
+// Hide tooltips on mobile
+@media (pointer:coarse) {
+  .tooltip::after {
+    display: none;
+  }
+}

bookmarks/styles/theme-dark.scss

@@ -12,6 +12,7 @@
 @import "bookmark-form";
 @import "settings";
 @import "markdown";
+@import "reader-mode";
 
 /* Dark theme overrides */
 

bookmarks/styles/theme-light.scss

@@ -12,3 +12,4 @@
 @import "bookmark-form";
 @import "settings";
 @import "markdown";
+@import "reader-mode";

bookmarks/tasks.py

@@ -0,0 +1,2 @@
+# Expose task modules to Huey Django extension
+import bookmarks.services.tasks

bookmarks/templates/admin/background_tasks.html

@@ -0,0 +1,39 @@
+{% extends "admin/base_site.html" %}
+
+{% block content %}
+  <table style="width: 100%">
+    <thead>
+    <tr>
+      <th>ID</th>
+      <th>Name</th>
+      <th>Args</th>
+      <th>Retries</th>
+    </tr>
+    </thead>
+    <tbody>
+    {% for task in tasks %}
+      <tr>
+        <td>{{ task.id }}</td>
+        <td>{{ task.name }}</td>
+        <td>{{ task.args }}</td>
+        <td>{{ task.retries }}</td>
+      </tr>
+    {% endfor %}
+    </tbody>
+  </table>
+  <p class="paginator">
+    {% if page.paginator.num_pages > 1 %}
+      {% for page_number in page_range %}
+        {% if page_number == page.number %}
+          <span class="this-page">{{ page_number }}</span>
+        {% elif page_number == '…' %}
+          <span>…</span>
+        {% else %}
+          <a href="?p={{ page_number }}">{{ page_number }}</a>
+        {% endif %}
+      {% endfor %}
+      &nbsp;
+    {% endif %}
+    {{ page.paginator.count }} tasks
+  </p>
+{% endblock %}

bookmarks/templates/bookmarks/archive.html

@@ -4,11 +4,7 @@
 {% load bookmarks %}
 
 {% block content %}
-  <div class="bookmarks-page grid columns-md-1"
+  <div ld-bulk-edit class="bookmarks-page grid columns-md-1">
-       ld-bulk-edit
-       ld-bookmark-page
-       bookmarks-url="{% url 'bookmarks:partials.bookmark_list.archived' %}"
-       tags-url="{% url 'bookmarks:partials.tag_cloud.archived' %}">
 
     {# Bookmark list #}
     <section class="content-area col-2">
@@ -17,17 +13,22 @@
         <div class="header-controls">
           {% bookmark_search bookmark_list.search tag_cloud.tags mode='archived' %}
           {% include 'bookmarks/bulk_edit/toggle.html' %}
-          <button ld-modal modal-content=".tag-cloud" class="btn ml-2 show-md">Tags</button>
+          <button ld-fetch="{{ bookmark_list.tag_modal_url }}" ld-target="body|append" ld-on="click"
+                  class="btn ml-2 show-md">Tags
+          </button>
         </div>
       </div>
 
-      <form class="bookmark-actions"
+      <form ld-form ld-fire="refresh-bookmark-list,refresh-tag-cloud"
+            class="bookmark-actions"
             action="{{ bookmark_list.action_url|safe }}"
             method="post" autocomplete="off">
         {% csrf_token %}
         {% include 'bookmarks/bulk_edit/bar.html' with disable_actions='bulk_archive' %}
 
-        <div class="bookmark-list-container">
+        <div ld-fetch="{{ bookmark_list.refresh_url }}" ld-on="refresh-bookmark-list"
+             ld-fire="refresh-bookmark-list-done"
+             class="bookmark-list-container">
           {% include 'bookmarks/bookmark_list.html' %}
         </div>
       </form>
@@ -38,7 +39,8 @@
       <div class="content-area-header">
         <h2>Tags</h2>
       </div>
-      <div class="tag-cloud-container">
+      <div ld-fetch="{{ tag_cloud.refresh_url }}" ld-on="refresh-tag-cloud"
+           class="tag-cloud-container">
         {% include 'bookmarks/tag_cloud.html' %}
       </div>
     </section>

bookmarks/templates/bookmarks/bookmark_list.html

@@ -10,142 +10,143 @@
       data-bookmarks-total="{{ bookmark_list.bookmarks_total }}">
     {% for bookmark_item in bookmark_list.items %}
       <li ld-bookmark-item{% if bookmark_item.css_classes %} class="{{ bookmark_item.css_classes }}"{% endif %}>
-        <div class="title">
+        <div class="content">
-          <label ld-bulk-edit-checkbox class="form-checkbox">
+          <div class="title">
-            <input type="checkbox" name="bookmark_id" value="{{ bookmark_item.id }}">
+            <label class="form-checkbox bulk-edit-checkbox">
-            <i class="form-icon"></i>
+              <input type="checkbox" name="bookmark_id" value="{{ bookmark_item.id }}">
-          </label>
+              <i class="form-icon"></i>
-          {% if bookmark_item.favicon_file and bookmark_list.show_favicons %}
+            </label>
-            <img src="{% static bookmark_item.favicon_file %}" alt="">
+            {% if bookmark_item.favicon_file and bookmark_list.show_favicons %}
-          {% endif %}
+              <img class="favicon" src="{% static bookmark_item.favicon_file %}" alt="">
-          <a href="{{ bookmark_item.url }}" target="{{ bookmark_list.link_target }}" rel="noopener">
+            {% endif %}
-            <span>{{ bookmark_item.title }}</span>
+            <a href="{{ bookmark_item.url }}" target="{{ bookmark_list.link_target }}" rel="noopener">
-          </a>
+              <span>{{ bookmark_item.title }}</span>
-        </div>
-        {% if bookmark_list.show_url %}
-          <div class="url-path truncate">
-            <a href="{{ bookmark_item.url }}" target="{{ bookmark_list.link_target }}" rel="noopener"
-               class="url-display">
-              {{ bookmark_item.url }}
             </a>
           </div>
-        {% endif %}
+          {% if bookmark_list.show_url %}
-        {% if bookmark_list.description_display == 'inline' %}
+            <div class="url-path truncate">
-          <div class="description inline truncate">
+              <a href="{{ bookmark_item.url }}" target="{{ bookmark_list.link_target }}" rel="noopener"
+                class="url-display">
+                {{ bookmark_item.url }}
+              </a>
+            </div>
+          {% endif %}
+          {% if bookmark_list.description_display == 'inline' %}
+            <div class="description inline truncate">
+              {% if bookmark_item.tag_names %}
+                <span class="tags">
+                  {% for tag_name in bookmark_item.tag_names %}
+                    <a href="?{% add_tag_to_query tag_name %}">{{ tag_name|hash_tag }}</a>
+                  {% endfor %}
+                </span>
+              {% endif %}
+              {% if bookmark_item.tag_names and bookmark_item.description %} | {% endif %}
+              {% if bookmark_item.description %}
+                <span>{{ bookmark_item.description }}</span>
+              {% endif %}
+            </div>
+          {% else %}
+            {% if bookmark_item.description %}
+              <div class="description separate">{{ bookmark_item.description }}</div>
+            {% endif %}
             {% if bookmark_item.tag_names %}
-              <span class="tags">
+              <div class="tags">
                 {% for tag_name in bookmark_item.tag_names %}
                   <a href="?{% add_tag_to_query tag_name %}">{{ tag_name|hash_tag }}</a>
                 {% endfor %}
+              </div>
+            {% endif %}
+          {% endif %}
+          {% if bookmark_item.notes %}
+            <div class="notes bg-gray text-gray-dark">
+              <div class="markdown">{% markdown bookmark_item.notes %}</div>
+            </div>
+          {% endif %}
+          <div class="actions text-gray">
+            {% if bookmark_item.display_date %}
+              {% if bookmark_item.web_archive_snapshot_url %}
+                <a href="{{ bookmark_item.web_archive_snapshot_url }}"
+                  title="Show snapshot on the Internet Archive Wayback Machine"
+                  target="{{ bookmark_list.link_target }}"
+                  rel="noopener">
+                  {{ bookmark_item.display_date }} ∞
+                </a>
+              {% else %}
+                <span>{{ bookmark_item.display_date }}</span>
+              {% endif %}
+              <span>|</span>
+            {% endif %}
+            {# View link is visible for both owned and shared bookmarks #}
+            {% if bookmark_list.show_view_action %}
+              <a ld-fetch="{% url 'bookmarks:details_modal' bookmark_item.id %}?return_url={{ bookmark_list.return_url|urlencode }}"
+                ld-on="click" ld-target="body|append"
+                href="{% url 'bookmarks:details' bookmark_item.id %}">View</a>
+            {% endif %}
+            {% if bookmark_item.is_editable %}
+              {# Bookmark owner actions #}
+              {% if bookmark_list.show_edit_action %}
+                <a href="{% url 'bookmarks:edit' bookmark_item.id %}?return_url={{ bookmark_list.return_url|urlencode }}">Edit</a>
+              {% endif %}
+              {% if bookmark_list.show_archive_action %}
+                {% if bookmark_item.is_archived %}
+                  <button type="submit" name="unarchive" value="{{ bookmark_item.id }}"
+                          class="btn btn-link btn-sm">Unarchive
+                  </button>
+                {% else %}
+                  <button type="submit" name="archive" value="{{ bookmark_item.id }}"
+                          class="btn btn-link btn-sm">Archive
+                  </button>
+                {% endif %}
+              {% endif %}
+              {% if bookmark_list.show_remove_action %}
+                <button ld-confirm-button type="submit" name="remove" value="{{ bookmark_item.id }}"
+                        class="btn btn-link btn-sm">Remove
+                </button>
+              {% endif %}
+            {% else %}
+              {# Shared bookmark actions #}
+              <span>Shared by
+                <a href="?{% replace_query_param user=bookmark_item.owner.username %}">{{ bookmark_item.owner.username }}</a>
               </span>
             {% endif %}
-            {% if bookmark_item.tag_names and bookmark_item.description %} | {% endif %}
+            {% if bookmark_item.has_extra_actions %}
-            {% if bookmark_item.description %}
+              <div class="extra-actions">
-              <span>{{ bookmark_item.description }}</span>
+                <span class="hide-sm">|</span>
+                {% if bookmark_item.show_mark_as_read %}
+                  <button type="submit" name="mark_as_read" value="{{ bookmark_item.id }}"
+                          class="btn btn-link btn-sm btn-icon"
+                          ld-confirm-button ld-confirm-icon="ld-icon-read" ld-confirm-question="Mark as read?">
+                    <svg xmlns="http://www.w3.org/2000/svg" width="16" height="16">
+                      <use xlink:href="#ld-icon-unread"></use>
+                    </svg>
+                    Unread
+                  </button>
+                {% endif %}
+                {% if bookmark_item.show_unshare %}
+                  <button type="submit" name="unshare" value="{{ bookmark_item.id }}"
+                          class="btn btn-link btn-sm btn-icon"
+                          ld-confirm-button ld-confirm-icon="ld-icon-unshare" ld-confirm-question="Unshare?">
+                    <svg xmlns="http://www.w3.org/2000/svg" width="16" height="16">
+                      <use xlink:href="#ld-icon-share"></use>
+                    </svg>
+                    Shared
+                  </button>
+                {% endif %}
+                {% if bookmark_item.show_notes_button %}
+                  <button type="button" class="btn btn-link btn-sm btn-icon toggle-notes">
+                    <svg xmlns="http://www.w3.org/2000/svg" width="16" height="16">
+                      <use xlink:href="#ld-icon-note"></use>
+                    </svg>
+                    Notes
+                  </button>
+                {% endif %}
+              </div>
             {% endif %}
           </div>
-        {% else %}
-          {% if bookmark_item.description %}
-            <div class="description separate">
-              {{ bookmark_item.description }}
-            </div>
-          {% endif %}
-          {% if bookmark_item.tag_names %}
-            <div class="tags">
-              {% for tag_name in bookmark_item.tag_names %}
-                <a href="?{% add_tag_to_query tag_name %}">{{ tag_name|hash_tag }}</a>
-              {% endfor %}
-            </div>
-          {% endif %}
-        {% endif %}
-        {% if bookmark_item.notes %}
-          <div class="notes bg-gray text-gray-dark">
-            <div class="markdown">
-              {% markdown bookmark_item.notes %}
-            </div>
-          </div>
-        {% endif %}
-        <div class="actions text-gray">
-          {% if bookmark_item.display_date %}
-            {% if bookmark_item.web_archive_snapshot_url %}
-              <a href="{{ bookmark_item.web_archive_snapshot_url }}"
-                 title="Show snapshot on the Internet Archive Wayback Machine"
-                 target="{{ bookmark_list.link_target }}"
-                 rel="noopener">
-                {{ bookmark_item.display_date }} ∞
-              </a>
-            {% else %}
-              <span>{{ bookmark_item.display_date }}</span>
-            {% endif %}
-            <span>|</span>
-          {% endif %}
-          {# View link is visible for both owned and shared bookmarks #}
-          {% if bookmark_list.show_view_action %}
-            <a ld-modal
-               modal-url="{% url 'bookmarks:details_modal' bookmark_item.id %}?return_url={{ bookmark_list.return_url|urlencode }}"
-               href="{% url 'bookmarks:details' bookmark_item.id %}">View</a>
-          {% endif %}
-          {% if bookmark_item.is_editable %}
-            {# Bookmark owner actions #}
-            {% if bookmark_list.show_edit_action %}
-              <a href="{% url 'bookmarks:edit' bookmark_item.id %}?return_url={{ bookmark_list.return_url|urlencode }}">Edit</a>
-            {% endif %}
-            {% if bookmark_list.show_archive_action %}
-              {% if bookmark_item.is_archived %}
-                <button type="submit" name="unarchive" value="{{ bookmark_item.id }}"
-                        class="btn btn-link btn-sm">Unarchive
-                </button>
-              {% else %}
-                <button type="submit" name="archive" value="{{ bookmark_item.id }}"
-                        class="btn btn-link btn-sm">Archive
-                </button>
-              {% endif %}
-            {% endif %}
-            {% if bookmark_list.show_remove_action %}
-              <button ld-confirm-button type="submit" name="remove" value="{{ bookmark_item.id }}"
-                      class="btn btn-link btn-sm">Remove
-              </button>
-            {% endif %}
-          {% else %}
-            {# Shared bookmark actions #}
-            <span>Shared by
-              <a href="?{% replace_query_param user=bookmark_item.owner.username %}">{{ bookmark_item.owner.username }}</a>
-            </span>
-          {% endif %}
-          {% if bookmark_item.has_extra_actions %}
-            <div class="extra-actions">
-              <span class="hide-sm">|</span>
-              {% if bookmark_item.show_mark_as_read %}
-                <button type="submit" name="mark_as_read" value="{{ bookmark_item.id }}"
-                        class="btn btn-link btn-sm btn-icon"
-                        ld-confirm-button confirm-icon="ld-icon-read" confirm-question="Mark as read?">
-                  <svg xmlns="http://www.w3.org/2000/svg" width="16" height="16">
-                    <use xlink:href="#ld-icon-unread"></use>
-                  </svg>
-                  Unread
-                </button>
-              {% endif %}
-              {% if bookmark_item.show_unshare %}
-                <button type="submit" name="unshare" value="{{ bookmark_item.id }}"
-                        class="btn btn-link btn-sm btn-icon"
-                        ld-confirm-button confirm-icon="ld-icon-unshare" confirm-question="Unshare?">
-                  <svg xmlns="http://www.w3.org/2000/svg" width="16" height="16">
-                    <use xlink:href="#ld-icon-share"></use>
-                  </svg>
-                  Shared
-                </button>
-              {% endif %}
-              {% if bookmark_item.show_notes_button %}
-                <button type="button" class="btn btn-link btn-sm btn-icon toggle-notes">
-                  <svg xmlns="http://www.w3.org/2000/svg" width="16" height="16">
-                    <use xlink:href="#ld-icon-note"></use>
-                  </svg>
-                  Notes
-                </button>
-              {% endif %}
-            </div>
-          {% endif %}
         </div>
+        {% if bookmark_list.show_preview_images and bookmark_item.preview_image_file %}
+          <img class="preview-image" src="{% static bookmark_item.preview_image_file %}" loading="lazy"/>
+        {% endif %}
       </li>
     {% endfor %}
   </ul>

bookmarks/templates/bookmarks/bulk_edit/bar.html

@@ -2,7 +2,7 @@
 {% htmlmin %}
   <div class="bulk-edit-bar">
     <div class="bulk-edit-actions bg-gray">
-      <label ld-bulk-edit-checkbox all class="form-checkbox">
+      <label class="form-checkbox bulk-edit-checkbox all">
         <input type="checkbox">
         <i class="form-icon"></i>
       </label>

bookmarks/templates/bookmarks/bulk_edit/toggle.html

@@ -1,4 +1,4 @@
-<button ld-bulk-edit-active-toggle class="btn hide-sm ml-2" title="Bulk edit">
+<button class="btn hide-sm ml-2 bulk-edit-active-toggle" title="Bulk edit">
   <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 20 20" fill="currentColor" width="20px"
        height="20px">
     <path

bookmarks/templates/bookmarks/details.html

@@ -1,13 +1,13 @@
 {% extends 'bookmarks/layout.html' %}
 
 {% block content %}
-  <div ld-bookmark-details class="bookmark-details page">
+  <div class="bookmark-details page">
-    {% if request.user == bookmark.owner %}
+    {% if details.is_editable %}
       {% include 'bookmarks/details/actions.html' %}
     {% endif %}
     {% include 'bookmarks/details/title.html' %}
     <div>
-      {% include 'bookmarks/details/content.html' %}
+      {% include 'bookmarks/details/form.html' %}
     </div>
   </div>
 {% endblock %}

bookmarks/templates/bookmarks/details/actions.html

@@ -1,11 +1,14 @@
 <div class="actions">
   <div class="left-actions">
-    <a class="btn" href="{% url 'bookmarks:edit' bookmark.id %}?return_url={{ edit_return_url|urlencode }}">Edit</a>
+    <a class="btn"
+       href="{% url 'bookmarks:edit' details.bookmark.id %}?return_url={{ details.edit_return_url|urlencode }}">Edit</a>
   </div>
   <div class="right-actions">
-    <form action="{% url 'bookmarks:index.action' %}?return_url={{ delete_return_url|urlencode }}" method="post">
+    <form action="{% url 'bookmarks:index.action' %}?return_url={{ details.delete_return_url|urlencode }}"
+          method="post">
       {% csrf_token %}
-      <button ld-confirm-button type="submit" name="remove" value="{{ bookmark.id }}" class="btn btn-link text-error">
+      <button ld-confirm-button type="submit" name="remove" value="{{ details.bookmark.id }}"
+              class="btn btn-link text-error">
         Delete...
       </button>
     </form>

bookmarks/templates/bookmarks/details/asset_icon.html

@@ -0,0 +1,42 @@
+{% if asset.content_type == 'text/html' %}
+  <svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none"
+       stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
+    <path stroke="none" d="M0 0h24v24H0z" fill="none"/>
+    <path d="M14 3v4a1 1 0 0 0 1 1h4"/>
+    <path d="M5 12v-7a2 2 0 0 1 2 -2h7l5 5v4"/>
+    <path d="M2 21v-6"/>
+    <path d="M5 15v6"/>
+    <path d="M2 18h3"/>
+    <path d="M20 15v6h2"/>
+    <path d="M13 21v-6l2 3l2 -3v6"/>
+    <path d="M7.5 15h3"/>
+    <path d="M9 15v6"/>
+  </svg>
+{% elif asset.content_type == 'application/pdf' %}
+  <svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none"
+       stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
+    <path stroke="none" d="M0 0h24v24H0z" fill="none"/>
+    <path d="M14 3v4a1 1 0 0 0 1 1h4"/>
+    <path d="M5 12v-7a2 2 0 0 1 2 -2h7l5 5v4"/>
+    <path d="M5 18h1.5a1.5 1.5 0 0 0 0 -3h-1.5v6"/>
+    <path d="M17 18h2"/>
+    <path d="M20 15h-3v6"/>
+    <path d="M11 15v6h1a2 2 0 0 0 2 -2v-2a2 2 0 0 0 -2 -2h-1z"/>
+  </svg>
+{% elif asset.content_type == 'image/png' or asset.content_type == 'image/jpeg' or asset.content_type == 'image.gif' %}
+  <svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none"
+       stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
+    <path stroke="none" d="M0 0h24v24H0z" fill="none"/>
+    <path d="M15 8h.01"/>
+    <path d="M3 6a3 3 0 0 1 3 -3h12a3 3 0 0 1 3 3v12a3 3 0 0 1 -3 3h-12a3 3 0 0 1 -3 -3v-12z"/>
+    <path d="M3 16l5 -5c.928 -.893 2.072 -.893 3 0l5 5"/>
+    <path d="M14 14l1 -1c.928 -.893 2.072 -.893 3 0l3 3"/>
+  </svg>
+{% else %}
+  <svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor"
+       stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
+    <path stroke="none" d="M0 0h24v24H0z" fill="none"/>
+    <path d="M14 3v4a1 1 0 0 0 1 1h4"/>
+    <path d="M17 21h-10a2 2 0 0 1 -2 -2v-14a2 2 0 0 1 2 -2h7l5 5v11a2 2 0 0 1 -2 2z"/>
+  </svg>
+{% endif %}

bookmarks/templates/bookmarks/details/assets.html

@@ -0,0 +1,48 @@
+<div {% if details.has_pending_assets %}
+  ld-fetch="{% url 'bookmarks:details_assets' details.bookmark.id %}"
+  ld-interval="5" ld-target="self|outerHTML"
+{% endif %}>
+  {% if details.assets %}
+    <div class="assets">
+      {% for asset in details.assets %}
+        <div class="asset" data-asset-id="{{ asset.id }}">
+          <div class="asset-icon {{ asset.icon_classes }}">
+            {% include 'bookmarks/details/asset_icon.html' %}
+          </div>
+          <div class="asset-text {{ asset.text_classes }}">
+          <span class="truncate">
+            {{ asset.display_name }}
+            {% if asset.status == 'pending' %}(queued){% endif %}
+            {% if asset.status == 'failure' %}(failed){% endif %}
+          </span>
+            {% if asset.file_size %}
+              <span class="filesize">{{ asset.file_size|filesizeformat }}</span>
+            {% endif %}
+          </div>
+          <div class="asset-actions">
+            {% if asset.file %}
+              <a class="btn btn-link" href="{% url 'bookmarks:assets.view' asset.id %}" target="_blank">View</a>
+            {% endif %}
+            {% if details.is_editable %}
+              <button ld-confirm-button type="submit" name="remove_asset" value="{{ asset.id }}" class="btn btn-link">
+                Remove
+              </button>
+            {% endif %}
+          </div>
+        </div>
+      {% endfor %}
+    </div>
+  {% endif %}
+
+  {% if details.is_editable %}
+    <div class="assets-actions">
+      <button type="submit" name="create_snapshot" class="btn btn-link"
+              {% if details.has_pending_assets %}disabled{% endif %}>Create HTML snapshot
+      </button>
+      <button ld-upload-button id="upload-asset" name="upload_asset" value="{{ details.bookmark.id }}" type="button"
+              class="btn btn-link">Upload file
+      </button>
+      <input id="upload-asset-file" name="upload_asset_file" type="file" class="d-hide">
+    </div>
+  {% endif %}
+</div>

bookmarks/templates/bookmarks/details/content.html

@@ -1,85 +0,0 @@
-{% load static %}
-{% load shared %}
-
-<div class="weblinks">
-  <a class="weblink" href="{{ bookmark.url }}" rel="noopener"
-     target="{{ request.user_profile.bookmark_link_target }}">
-    {% if bookmark.favicon_file and request.user_profile.enable_favicons %}
-      <img class="favicon" src="{% static bookmark.favicon_file %}" alt="">
-    {% endif %}
-    <span>{{ bookmark.url }}</span>
-  </a>
-  {% if bookmark.web_archive_snapshot_url %}
-    <a class="weblink" href="{{ bookmark.web_archive_snapshot_url }}"
-       target="{{ request.user_profile.bookmark_link_target }}">
-      {% if bookmark.favicon_file and request.user_profile.enable_favicons %}
-        <svg class="favicon" viewBox="0 0 76 86" xmlns="http://www.w3.org/2000/svg">
-          <path
-              d="m76 82v4h-76l.00080851-4zm-3-6v5h-70v-5zm-62.6696277-54 .8344146.4217275.4176066 6.7436084.4176065 10.9576581v10.5383496l-.4176065 13.1364492-.0694681 8.8498268-1.1825531.3523804h-4.17367003l-1.25202116-.3523804-.48627608-8.8498268-.41840503-13.0662957v-10.5375432l.41840503-11.028618.38167482-6.7798947.87034634-.3854412zm60.0004653 0 .8353798.4217275.4168913 6.7436084.4168913 10.9576581v10.5383496l-.4168913 13.1364492-.0686832 8.8498268-1.1835879.3523804h-4.1737047l-1.2522712-.3523804-.4879704-8.8498268-.4168913-13.0662957v-10.5375432l.4168913-11.028618.3833483-6.7798947.8697215-.3854412zm-42.000632 0 .8344979.4217275.4176483 6.7436084.4176482 10.9576581v10.5383496l-.4176482 13.1364492-.0686764 8.8498268-1.1834698.3523804h-4.1740866l-1.2529447-.3523804-.4863246-8.8498268-.4168497-13.0662957v-10.5375432l.4168497-11.028618.38331-6.7798947.8688361-.3854412zm23 0 .8344979.4217275.4176483 6.7436084.4176482 10.9576581v10.5383496l-.4176482 13.1364492-.0686764 8.8498268-1.1834698.3523804h-4.1740866l-1.2521462-.3523804-.4871231-8.8498268-.4168497-13.0662957v-10.5375432l.4168497-11.028618.38331-6.7798947.8696347-.3854412zm21.6697944-9v7h-70v-7zm-35.7200748-13 36.7200748 8.4088317-1.4720205 2.5911683h-70.32799254l-2.19998696-2.10140371z"
-              fill="currentColor" fill-rule="evenodd"/>
-        </svg>
-      {% endif %}
-      <span>View on Internet Archive</span>
-    </a>
-  {% endif %}
-</div>
-<dl class="grid columns-2 columns-sm-1 gap-0">
-  {% if request.user == bookmark.owner %}
-    <div class="status col-2">
-      <dt>Status</dt>
-      <dd class="d-flex" style="gap: .8rem">
-        <form action="{% url 'bookmarks:details' bookmark.id %}" method="post">
-          {% csrf_token %}
-          <div class="form-group">
-            <label class="form-switch">
-              <input type="checkbox" name="is_archived" {% if bookmark.is_archived %}checked{% endif %}>
-              <i class="form-icon"></i> Archived
-            </label>
-          </div>
-          <div class="form-group">
-            <label class="form-switch">
-              <input type="checkbox" name="unread" {% if bookmark.unread %}checked{% endif %}>
-              <i class="form-icon"></i> Unread
-            </label>
-          </div>
-          {% if request.user_profile.enable_sharing %}
-            <div class="form-group">
-              <label class="form-switch">
-                <input type="checkbox" name="shared" {% if bookmark.shared %}checked{% endif %}>
-                <i class="form-icon"></i> Shared
-              </label>
-            </div>
-          {% endif %}
-        </form>
-      </dd>
-    </div>
-  {% endif %}
-  {% if bookmark.tag_names %}
-    <div class="tags col-1">
-      <dt>Tags</dt>
-      <dd>
-        {% for tag_name in bookmark.tag_names %}
-          <a href="{% url 'bookmarks:index' %}?{% add_tag_to_query tag_name %}">{{ tag_name|hash_tag }}</a>
-        {% endfor %}
-      </dd>
-    </div>
-  {% endif %}
-  <div class="date-added col-1">
-    <dt>Date added</dt>
-    <dd>
-      <span>{{ bookmark.date_added }}</span>
-    </dd>
-  </div>
-  {% if bookmark.resolved_description %}
-    <div class="description col-2">
-      <dt>Description</dt>
-      <dd>{{ bookmark.resolved_description }}</dd>
-    </div>
-  {% endif %}
-  {% if bookmark.notes %}
-    <div class="notes col-2">
-      <dt>Notes</dt>
-      <dd class="markdown">{% markdown bookmark.notes %}</dd>
-    </div>
-  {% endif %}
-</dl>

bookmarks/templates/bookmarks/details/form.html

@@ -0,0 +1,114 @@
+{% load static %}
+{% load shared %}
+
+<form ld-form ld-fire="refresh-bookmark-list,refresh-tag-cloud,refresh-details"
+      action="{% url 'bookmarks:details' details.bookmark.id %}"
+      method="post">
+  <div class="weblinks">
+    <a class="weblink" href="{{ details.bookmark.url }}" rel="noopener"
+       target="{{ details.profile.bookmark_link_target }}">
+      {% if details.show_link_icons %}
+        <img class="favicon" src="{% static details.bookmark.favicon_file %}" alt="">
+      {% endif %}
+      <span>{{ details.bookmark.url }}</span>
+    </a>
+    {% if details.latest_snapshot %}
+      <a class="weblink" href="{% url 'bookmarks:assets.read' details.latest_snapshot.id %}"
+         target="{{ details.profile.bookmark_link_target }}">
+        {% if details.show_link_icons %}
+          <svg class="favicon" xmlns="http://www.w3.org/2000/svg">
+            <use xlink:href="#ld-icon-unread"></use>
+          </svg>
+        {% endif %}
+        <span>Reader mode</span>
+      </a>
+    {% endif %}
+    {% if details.web_archive_snapshot_url %}
+      <a class="weblink" href="{{ details.web_archive_snapshot_url }}"
+         target="{{ details.profile.bookmark_link_target }}">
+        {% if details.show_link_icons %}
+          <svg class="favicon" viewBox="0 0 76 86" xmlns="http://www.w3.org/2000/svg">
+            <path
+                d="m76 82v4h-76l.00080851-4zm-3-6v5h-70v-5zm-62.6696277-54 .8344146.4217275.4176066 6.7436084.4176065 10.9576581v10.5383496l-.4176065 13.1364492-.0694681 8.8498268-1.1825531.3523804h-4.17367003l-1.25202116-.3523804-.48627608-8.8498268-.41840503-13.0662957v-10.5375432l.41840503-11.028618.38167482-6.7798947.87034634-.3854412zm60.0004653 0 .8353798.4217275.4168913 6.7436084.4168913 10.9576581v10.5383496l-.4168913 13.1364492-.0686832 8.8498268-1.1835879.3523804h-4.1737047l-1.2522712-.3523804-.4879704-8.8498268-.4168913-13.0662957v-10.5375432l.4168913-11.028618.3833483-6.7798947.8697215-.3854412zm-42.000632 0 .8344979.4217275.4176483 6.7436084.4176482 10.9576581v10.5383496l-.4176482 13.1364492-.0686764 8.8498268-1.1834698.3523804h-4.1740866l-1.2529447-.3523804-.4863246-8.8498268-.4168497-13.0662957v-10.5375432l.4168497-11.028618.38331-6.7798947.8688361-.3854412zm23 0 .8344979.4217275.4176483 6.7436084.4176482 10.9576581v10.5383496l-.4176482 13.1364492-.0686764 8.8498268-1.1834698.3523804h-4.1740866l-1.2521462-.3523804-.4871231-8.8498268-.4168497-13.0662957v-10.5375432l.4168497-11.028618.38331-6.7798947.8696347-.3854412zm21.6697944-9v7h-70v-7zm-35.7200748-13 36.7200748 8.4088317-1.4720205 2.5911683h-70.32799254l-2.19998696-2.10140371z"
+                fill="currentColor" fill-rule="evenodd"/>
+          </svg>
+        {% endif %}
+        <span>Internet Archive</span>
+      </a>
+    {% endif %}
+  </div>
+  {% if details.preview_image_enabled and details.bookmark.preview_image_file %}
+    <div class="preview-image">
+      <img src="{% static details.bookmark.preview_image_file %}"/>
+    </div>
+  {% endif %}
+  <dl class="grid columns-2 columns-sm-1 gap-0">
+    {% if details.is_editable %}
+      <div class="status col-2">
+        <dt>Status</dt>
+        <dd class="d-flex" style="gap: .8rem">
+          {% csrf_token %}
+          <div class="form-group">
+            <label class="form-switch">
+              <input ld-auto-submit type="checkbox" name="is_archived"
+                     {% if details.bookmark.is_archived %}checked{% endif %}>
+              <i class="form-icon"></i> Archived
+            </label>
+          </div>
+          <div class="form-group">
+            <label class="form-switch">
+              <input ld-auto-submit type="checkbox" name="unread"
+                     {% if details.bookmark.unread %}checked{% endif %}>
+              <i class="form-icon"></i> Unread
+            </label>
+          </div>
+          {% if details.profile.enable_sharing %}
+            <div class="form-group">
+              <label class="form-switch">
+                <input ld-auto-submit type="checkbox" name="shared"
+                       {% if details.bookmark.shared %}checked{% endif %}>
+                <i class="form-icon"></i> Shared
+              </label>
+            </div>
+          {% endif %}
+        </dd>
+      </div>
+    {% endif %}
+    {% if details.show_files %}
+      <div class="files col-2">
+        <dt>Files</dt>
+        <dd>
+          {% include 'bookmarks/details/assets.html' %}
+        </dd>
+      </div>
+    {% endif %}
+    {% if details.bookmark.tag_names %}
+      <div class="tags col-1">
+        <dt>Tags</dt>
+        <dd>
+          {% for tag_name in details.bookmark.tag_names %}
+            <a href="{% url 'bookmarks:index' %}?{% add_tag_to_query tag_name %}">{{ tag_name|hash_tag }}</a>
+          {% endfor %}
+        </dd>
+      </div>
+    {% endif %}
+    <div class="date-added col-1">
+      <dt>Date added</dt>
+      <dd>
+        <span>{{ details.bookmark.date_added }}</span>
+      </dd>
+    </div>
+    {% if details.bookmark.resolved_description %}
+      <div class="description col-2">
+        <dt>Description</dt>
+        <dd>{{ details.bookmark.resolved_description }}</dd>
+      </div>
+    {% endif %}
+    {% if details.bookmark.notes %}
+      <div class="notes col-2">
+        <dt>Notes</dt>
+        <dd class="markdown">{% markdown details.bookmark.notes %}</dd>
+      </div>
+    {% endif %}
+  </dl>
+</form>

bookmarks/templates/bookmarks/details/title.html

@@ -1,3 +1,3 @@
 <h2>
-  {{ bookmark.resolved_title }}
+  {{ details.bookmark.resolved_title }}
 </h2>

bookmarks/templates/bookmarks/details_modal.html

@@ -1,4 +1,7 @@
-<div ld-bookmark-details class="modal active bookmark-details">
+<div ld-modal
+     ld-fetch="{% url 'bookmarks:details_modal' details.bookmark.id %}" ld-on="refresh-details"
+     ld-select=".content" ld-target=".modal.bookmark-details .content|outerHTML"
+     class="modal active bookmark-details">
   <div class="modal-overlay" aria-label="Close"></div>
   <div class="modal-container">
     <div class="modal-header">
@@ -14,11 +17,11 @@
     </div>
     <div class="modal-body">
       <div class="content">
-        {% include 'bookmarks/details/content.html' %}
+        {% include 'bookmarks/details/form.html' %}
       </div>
     </div>
 
-    {% if request.user == bookmark.owner %}
+    {% if details.is_editable %}
       <div class="modal-footer">
         {% include 'bookmarks/details/actions.html' %}
       </div>

bookmarks/templates/bookmarks/form.html

@@ -23,10 +23,10 @@
     <label for="{{ form.tag_string.id_for_label }}" class="form-label">Tags</label>
     {{ form.tag_string|add_class:"form-input"|attr:"ld-tag-autocomplete"|attr:"autocomplete:off"|attr:"autocapitalize:off" }}
     <div class="form-input-hint">
-      Enter any number of tags separated by space and <strong>without</strong> the hash (#). If a tag does not
+      Enter any number of tags separated by space and <strong>without</strong> the hash (#).
-      exist it will be
+      If a tag does not exist it will be automatically created.
-      automatically created.
     </div>
+    <div class="form-input-hint auto-tags"></div>
     {{ form.tag_string.errors }}
   </div>
   <div class="form-group has-icon-right">
@@ -197,6 +197,18 @@
             } else {
               bookmarkExistsHint.style['display'] = 'none';
             }
+
+            // Preview auto tags
+            const autoTags = data.auto_tags;
+            const autoTagsHint = document.querySelector('.form-input-hint.auto-tags');
+
+            if (autoTags.length > 0) {
+              autoTags.sort();
+              autoTagsHint.style['display'] = 'block';
+              autoTagsHint.innerHTML = `Auto tags: ${autoTags.join(" ")}`;
+            } else {
+              autoTagsHint.style['display'] = 'none';
+            }
           });
       }
 

bookmarks/templates/bookmarks/index.html

@@ -4,11 +4,7 @@
 {% load bookmarks %}
 
 {% block content %}
-  <div class="bookmarks-page grid columns-md-1"
+  <div ld-bulk-edit class="bookmarks-page grid columns-md-1">
-       ld-bulk-edit
-       ld-bookmark-page
-       bookmarks-url="{% url 'bookmarks:partials.bookmark_list.active' %}"
-       tags-url="{% url 'bookmarks:partials.tag_cloud.active' %}">
 
     {# Bookmark list #}
     <section class="content-area col-2">
@@ -17,17 +13,22 @@
         <div class="header-controls">
           {% bookmark_search bookmark_list.search tag_cloud.tags %}
           {% include 'bookmarks/bulk_edit/toggle.html' %}
-          <button ld-modal modal-content=".tag-cloud" class="btn ml-2 show-md">Tags</button>
+          <button ld-fetch="{{ bookmark_list.tag_modal_url }}" ld-target="body|append" ld-on="click"
+                  class="btn ml-2 show-md">Tags
+          </button>
         </div>
       </div>
 
-      <form class="bookmark-actions"
+      <form ld-form ld-fire="refresh-bookmark-list,refresh-tag-cloud"
+            class="bookmark-actions"
             action="{{ bookmark_list.action_url|safe }}"
             method="post" autocomplete="off">
         {% csrf_token %}
         {% include 'bookmarks/bulk_edit/bar.html' with disable_actions='bulk_unarchive' %}
 
-        <div class="bookmark-list-container">
+        <div ld-fetch="{{ bookmark_list.refresh_url }}" ld-on="refresh-bookmark-list"
+             ld-fire="refresh-bookmark-list-done"
+             class="bookmark-list-container">
           {% include 'bookmarks/bookmark_list.html' %}
         </div>
       </form>
@@ -38,7 +39,8 @@
       <div class="content-area-header">
         <h2>Tags</h2>
       </div>
-      <div class="tag-cloud-container">
+      <div ld-fetch="{{ tag_cloud.refresh_url }}" ld-on="refresh-tag-cloud"
+           class="tag-cloud-container">
         {% include 'bookmarks/tag_cloud.html' %}
       </div>
     </section>

bookmarks/templates/bookmarks/layout.html

@@ -114,16 +114,16 @@
     </div>
   {% endif %}
   <div class="d-flex justify-between">
-    <a href="{% url 'bookmarks:index' %}" class="d-flex align-center">
+    <a href="{% url 'bookmarks:root' %}" class="d-flex align-center">
       <img class="logo" src="{% static 'logo.png' %}" alt="Application logo">
       <h1>LINKDING</h1>
     </a>
     {% if request.user.is_authenticated %}
       {# Only show nav items menu when logged in #}
       {% include 'bookmarks/nav_menu.html' %}
-    {% elif has_public_shares %}
+    {% else %}
-      {# Otherwise show link to shared bookmarks if there are publicly shared bookmarks #}
+      {# Otherwise show login link #}
-      <a href="{% url 'bookmarks:shared' %}" class="btn btn-link">Shared bookmarks</a>
+      <a href="{% url 'login' %}" class="btn btn-link">Login</a>
     {% endif %}
   </div>
 </header>

bookmarks/templates/bookmarks/read.html

@@ -0,0 +1,83 @@
+{% load sass_tags %}
+{% load static %}
+<!DOCTYPE html>
+<html lang="en" class="reader-mode">
+<head>
+  <meta charset="UTF-8">
+  <title>Reader view</title>
+  <meta name="viewport" content="width=device-width, initial-scale=1.0, minimal-ui">
+  {% if request.user_profile.theme == 'light' %}
+    <link href="{% sass_src 'theme-light.scss' %}?v={{ app_version }}" rel="stylesheet" type="text/css"/>
+  {% elif request.user_profile.theme == 'dark' %}
+    <link href="{% sass_src 'theme-dark.scss' %}?v={{ app_version }}" rel="stylesheet" type="text/css"/>
+  {% else %}
+    {# Use auto theme as fallback #}
+    <link href="{% sass_src 'theme-dark.scss' %}?v={{ app_version }}" rel="stylesheet" type="text/css"
+          media="(prefers-color-scheme: dark)"/>
+    <link href="{% sass_src 'theme-light.scss' %}?v={{ app_version }}" rel="stylesheet" type="text/css"
+          media="(prefers-color-scheme: light)"/>
+  {% endif %}
+</head>
+<body>
+<template id="content">{{ content|safe }}</template>
+<script src="{% static 'vendor/Readability.js' %}" type="application/javascript"></script>
+<script type="application/javascript">
+  function estimateReadingTime(charCount, wordsPerMinute) {
+    const avgWordLength = 5;
+    const totalWords = charCount / avgWordLength;
+    return Math.ceil(totalWords / wordsPerMinute);
+  }
+
+  function postProcess(articleContent) {
+    articleContent.querySelectorAll('table').forEach(table => {
+      table.classList.add('table');
+    });
+  }
+
+  function makeReadable() {
+    const content = document.getElementById('content');
+    const contentHtml = content.innerHTML;
+    const dom = new DOMParser().parseFromString(contentHtml, 'text/html');
+    const article = new Readability(dom).parse();
+
+    document.title = article.title;
+
+    const container = document.createElement('div');
+    container.classList.add('container');
+
+    const articleTitle = document.createElement('h1');
+    articleTitle.textContent = article.title;
+    container.append(articleTitle);
+
+    const byline = [article.byline, article.siteName].filter(Boolean);
+    if (byline.length > 0) {
+      const articleByline = document.createElement('p');
+      articleByline.textContent = byline.join(' | ');
+      articleByline.classList.add('byline');
+      container.append(articleByline);
+    }
+
+    if(article.length) {
+      const minTime = estimateReadingTime(article.length, 225);
+      const maxTime = estimateReadingTime(article.length, 175);
+
+      const articleReadingTime = document.createElement('p');
+      articleReadingTime.textContent = `${minTime}-${maxTime} minutes`;
+      articleReadingTime.classList.add('reading-time');
+      container.append(articleReadingTime);
+    }
+
+    const divider = document.createElement('hr');
+    container.append(divider);
+
+    const articleContent = document.createElement('div');
+    articleContent.innerHTML = article.content;
+    postProcess(articleContent);
+    container.append(articleContent);
+
+    content.replaceWith(container);
+  }
+  makeReadable();
+</script>
+</body>
+</html>

bookmarks/templates/bookmarks/shared.html

@@ -4,10 +4,7 @@
 {% load bookmarks %}
 
 {% block content %}
-  <div class="bookmarks-page grid columns-md-1"
+  <div class="bookmarks-page grid columns-md-1">
-       ld-bookmark-page
-       bookmarks-url="{% url 'bookmarks:partials.bookmark_list.shared' %}"
-       tags-url="{% url 'bookmarks:partials.tag_cloud.shared' %}">
 
     {# Bookmark list #}
     <section class="content-area col-2">
@@ -15,15 +12,20 @@
         <h2>Shared bookmarks</h2>
         <div class="header-controls">
           {% bookmark_search bookmark_list.search tag_cloud.tags mode='shared' %}
-          <button ld-modal modal-content=".tag-cloud" class="btn ml-2 show-md">Tags</button>
+          <button ld-fetch="{{ bookmark_list.tag_modal_url }}" ld-target="body|append" ld-on="click"
+                  class="btn ml-2 show-md">Tags
+          </button>
         </div>
       </div>
 
-      <form class="bookmark-actions" action="{{ bookmark_list.action_url|safe }}"
+      <form ld-form ld-fire="refresh-bookmark-list,refresh-tag-cloud"
-            method="post">
+            class="bookmark-actions"
+            action="{{ bookmark_list.action_url|safe }}"
+            method="post" autocomplete="off">
         {% csrf_token %}
-
+        <div ld-fetch="{{ bookmark_list.refresh_url }}" ld-on="refresh-bookmark-list"
-        <div class="bookmark-list-container">
+             ld-fire="refresh-bookmark-list-done"
+             class="bookmark-list-container">
           {% include 'bookmarks/bookmark_list.html' %}
         </div>
       </form>
@@ -41,7 +43,8 @@
       <div class="content-area-header">
         <h2>Tags</h2>
       </div>
-      <div class="tag-cloud-container">
+      <div ld-fetch="{{ tag_cloud.refresh_url }}" ld-on="refresh-tag-cloud"
+           class="tag-cloud-container">
         {% include 'bookmarks/tag_cloud.html' %}
       </div>
     </section>

bookmarks/templates/bookmarks/tag_modal.html

@@ -0,0 +1,21 @@
+<div ld-modal class="modal active">
+  <div class="modal-overlay" aria-label="Close"></div>
+  <div class="modal-container">
+    <div class="modal-header d-flex justify-between align-center">
+      <div class="modal-title h5">Tags</div>
+      <button class="close">
+        <svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" stroke-width="2"
+             stroke="currentColor" fill="none" stroke-linecap="round" stroke-linejoin="round">
+          <path stroke="none" d="M0 0h24v24H0z" fill="none"></path>
+          <path d="M18 6l-12 12"></path>
+          <path d="M6 6l12 12"></path>
+        </svg>
+      </button>
+    </div>
+    <div class="modal-body">
+      <div class="content">
+        {% include 'bookmarks/tag_cloud.html' %}
+      </div>
+    </div>
+  </div>
+</div>

bookmarks/templates/settings/general.html

@@ -7,6 +7,13 @@
     {% include 'settings/nav.html' %}
 
     {# Profile section #}
+    {% if success_message %}
+      <div class="toast toast-success mb-4">{{ success_message }}</div>
+    {% endif %}
+    {% if error_message %}
+      <div class="toast toast-error mb-4">{{ error_message }}</div>
+    {% endif %}
+
     <section class="content-area">
       <h2>Profile</h2>
       <p>
@@ -104,6 +111,29 @@
             result will also include bookmarks where a search term matches otherwise.
           </div>
         </div>
+        <div class="form-group">
+          <label for="{{ form.tag_grouping.id_for_label }}" class="form-label">Tag grouping</label>
+          {{ form.tag_grouping|add_class:"form-select width-25 width-sm-100" }}
+          <div class="form-input-hint">
+            In alphabetical mode, tags will be grouped by the first letter.
+            If disabled, tags will not be grouped.
+          </div>
+        </div>
+        <div class="form-group">
+          <details {% if form.auto_tagging_rules.value %}open{% endif %}>
+            <summary>Auto Tagging</summary>
+            <label for="{{ form.auto_tagging_rules.id_for_label }}" class="text-assistive">Auto Tagging</label>
+            <div class="mt-2">
+              {{ form.auto_tagging_rules|add_class:"form-input monospace"|attr:"rows:6" }}
+            </div>
+          </details>
+          <div class="form-input-hint">
+            Automatically adds tags to bookmarks based on predefined rules.
+            Each line is a single rule that maps a URL to one or more tags. For example:
+            <pre>youtube.com video
+reddit.com/r/Music music reddit</pre>
+          </div>
+        </div>
         <div class="form-group">
           <label for="{{ form.enable_favicons.id_for_label }}" class="form-checkbox">
             {{ form.enable_favicons }}
@@ -111,6 +141,7 @@
           </label>
           <div class="form-input-hint">
             Automatically loads favicons for bookmarked websites and displays them next to each bookmark.
+            Enabling this feature automatically downloads all missing favicons.
             By default, this feature uses a <b>Google service</b> to download favicons.
             If you don't want to use this service, check the <a
               href="https://github.com/sissbruecker/linkding/blob/master/docs/Options.md#ld_favicon_provider"
@@ -120,13 +151,16 @@
           {% if request.user_profile.enable_favicons and enable_refresh_favicons %}
             <button class="btn mt-2" name="refresh_favicons">Refresh Favicons</button>
           {% endif %}
-          {% if refresh_favicons_success_message %}
+        </div>
-            <div class="has-success">
+        <div class="form-group">
-              <p class="form-input-hint">
+          <label for="{{ form.enable_preview_images.id_for_label }}" class="form-checkbox">
-                {{ refresh_favicons_success_message }}
+            {{ form.enable_preview_images }}
-              </p>
+            <i class="form-icon"></i> Enable Preview Images
-            </div>
+          </label>
-          {% endif %}
+          <div class="form-input-hint">
+            Automatically loads preview images for bookmarked websites and displays them next to each bookmark.
+            Enabling this feature automatically downloads all missing preview images.
+          </div>
         </div>
         <div class="form-group">
           <label for="{{ form.web_archive_integration.id_for_label }}" class="form-label">Internet Archive
@@ -163,12 +197,36 @@
               href="{% url 'bookmarks:shared' %}">shared bookmarks page</a>.
           </div>
         </div>
+        {% if has_snapshot_support %}
+          <div class="form-group">
+            <label for="{{ form.enable_automatic_html_snapshots.id_for_label }}" class="form-checkbox">
+              {{ form.enable_automatic_html_snapshots }}
+              <i class="form-icon"></i> Automatically create HTML snapshots
+            </label>
+            <div class="form-input-hint">
+              Automatically creates HTML snapshots when adding bookmarks. Alternatively, when disabled, snapshots can be
+              created manually in the details view of a bookmark.
+            </div>
+            <button class="btn mt-2" name="create_missing_html_snapshots">Create missing HTML snapshots</button>
+          </div>
+        {% endif %}
+        <div class="form-group">
+          <label for="{{ form.default_mark_unread.id_for_label }}" class="form-checkbox">
+            {{ form.default_mark_unread }}
+            <i class="form-icon"></i> Create bookmarks as unread by default
+          </label>
+          <div class="form-input-hint">
+            Sets the default state for the "Mark as unread" option when creating a new bookmark.
+            Setting this option will make all new bookmarks default to unread.
+            This can be overridden when creating each new bookmark.
+          </div>
+        </div>
         <div class="form-group">
           <details {% if form.custom_css.value %}open{% endif %}>
             <summary>Custom CSS</summary>
             <label for="{{ form.custom_css.id_for_label }}" class="text-assistive">Custom CSS</label>
             <div class="mt-2">
-              {{ form.custom_css|add_class:"form-input custom-css"|attr:"rows:6" }}
+              {{ form.custom_css|add_class:"form-input monospace"|attr:"rows:6" }}
             </div>
           </details>
           <div class="form-input-hint">
@@ -177,17 +235,41 @@
         </div>
         <div class="form-group">
           <input type="submit" name="update_profile" value="Save" class="btn btn-primary mt-2">
-          {% if update_profile_success_message %}
-            <div class="has-success">
-              <p class="form-input-hint">
-                {{ update_profile_success_message }}
-              </p>
-            </div>
-          {% endif %}
         </div>
       </form>
     </section>
 
+    {# Global settings section #}
+    {% if global_settings_form %}
+      <section class="content-area">
+        <h2>Global settings</h2>
+        <form action="{% url 'bookmarks:settings.general' %}" method="post" novalidate>
+          {% csrf_token %}
+          <div class="form-group">
+            <label for="{{ global_settings_form.landing_page.id_for_label }}" class="form-label">Landing page</label>
+            {{ global_settings_form.landing_page|add_class:"form-select width-25 width-sm-100" }}
+            <div class="form-input-hint">
+              The page that unauthenticated users are redirected to when accessing the root URL.
+            </div>
+          </div>
+          <div class="form-group">
+            <label for="{{ global_settings_form.guest_profile_user.id_for_label }}" class="form-label">Guest user
+              profile</label>
+            {{ global_settings_form.guest_profile_user|add_class:"form-select width-25 width-sm-100" }}
+            <div class="form-input-hint">
+              The user profile to use for users that are not logged in. This will affect how publicly shared bookmarks
+              are displayed regarding theme, bookmark list settings, etc. You can either use your own profile or create
+              a dedicated user for this purpose. By default, a standard profile with fixed settings is used.
+            </div>
+          </div>
+
+          <div class="form-group">
+            <input type="submit" name="update_global_settings" value="Save" class="btn btn-primary mt-2">
+          </div>
+        </form>
+      </section>
+    {% endif %}
+
     {# Import section #}
     <section class="content-area">
       <h2>Import</h2>
@@ -212,20 +294,6 @@
             <input class="form-input" type="file" name="import_file">
             <input type="submit" class="input-group-btn btn btn-primary" value="Upload">
           </div>
-          {% if import_success_message %}
-            <div class="has-success">
-              <p class="form-input-hint">
-                {{ import_success_message }}
-              </p>
-            </div>
-          {% endif %}
-          {% if import_errors_message %}
-            <div class="has-error">
-              <p class="form-input-hint">
-                {{ import_errors_message }}
-              </p>
-            </div>
-          {% endif %}
         </div>
       </form>
     </section>

bookmarks/templates/settings/integrations.html

@@ -1,70 +1,90 @@
 {% extends "bookmarks/layout.html" %}
 
 {% block content %}
-    <div class="settings-page">
+  <div class="settings-page">
 
-        {% include 'settings/nav.html' %}
+    {% include 'settings/nav.html' %}
 
-        <section class="content-area">
+    <section class="content-area">
-            <h2>Browser Extension</h2>
+      <h2>Browser Extension</h2>
-            <p>The browser extension allows you to quickly add new bookmarks without leaving the page that you are on. The extension is available in the official extension stores for:</p>
+      <p>The browser extension allows you to quickly add new bookmarks without leaving the page that you are on. The
-            <ul>
+        extension is available in the official extension stores for:</p>
-                <li><a href="https://addons.mozilla.org/firefox/addon/linkding-extension/" target="_blank">Firefox</a></li>
+      <ul>
-                <li><a href="https://chrome.google.com/webstore/detail/linkding-extension/beakmhbijpdhipnjhnclmhgjlddhidpe" target="_blank">Chrome</a></li>
+        <li><a href="https://addons.mozilla.org/firefox/addon/linkding-extension/" target="_blank">Firefox</a></li>
-            </ul>
+        <li><a href="https://chrome.google.com/webstore/detail/linkding-extension/beakmhbijpdhipnjhnclmhgjlddhidpe"
-            <p>The extension is <a href="https://github.com/sissbruecker/linkding-extension" target="_blank">open source</a> as well, which enables you to build and manually load it into any browser that supports Chrome extensions.</p>
+               target="_blank">Chrome</a></li>
-            <h2>Bookmarklet</h2>
+      </ul>
-            <p>The bookmarklet is an alternative, cross-browser way to quickly add new bookmarks without opening the linkding application
+      <p>The extension is <a href="https://github.com/sissbruecker/linkding-extension" target="_blank">open source</a>
-                first. Here's how it works:</p>
+        as well, which enables you to build and manually load it into any browser that supports Chrome extensions.</p>
-            <ul>
+      <h2>Bookmarklet</h2>
-                <li>Drag the bookmarklet below into your browsers bookmark bar / toolbar</li>
+      <p>The bookmarklet is an alternative, cross-browser way to quickly add new bookmarks without opening the linkding
-                <li>Open the website that you want to bookmark</li>
+        application first. Here's how it works:</p>
-                <li>Click the bookmarklet in your browsers toolbar</li>
+      <ul>
-                <li>linkding opens in a new window or tab and allows you to add a bookmark for the site</li>
+        <li>Drag the bookmarklet below into your browsers bookmark bar / toolbar</li>
-                <li>After saving the bookmark the linkding window closes and you are back on your website</li>
+        <li>Open the website that you want to bookmark</li>
-            </ul>
+        <li>Click the bookmarklet in your browsers toolbar</li>
-            <p>Drag the following bookmarklet to your browsers toolbar:</p>
+        <li>linkding opens in a new window or tab and allows you to add a bookmark for the site</li>
-            <a href="javascript: {% include 'bookmarks/bookmarklet.js' %}"
+        <li>After saving the bookmark the linkding window closes and you are back on your website</li>
-               class="btn btn-primary">📎 Add bookmark</a>
+      </ul>
-        </section>
+      <p>Drag the following bookmarklet to your browser's toolbar:</p>
+      <a href="javascript: {% include 'bookmarks/bookmarklet.js' %}"
+         class="btn btn-primary">📎 Add bookmark</a>
+    </section>
 
-        <section class="content-area">
+    <section class="content-area">
-            <h2>REST API</h2>
+      <h2>REST API</h2>
-            <p>The following token can be used to authenticate 3rd-party applications against the REST API:</p>
+      <p>The following token can be used to authenticate 3rd-party applications against the REST API:</p>
-            <div class="form-group">
+      <div class="form-group">
-                <div class="columns">
+        <div class="columns">
-                    <div class="column width-50 width-md-100">
+          <div class="column width-50 width-md-100">
-                        <input class="form-input" value="{{ api_token }}" readonly>
+            <input class="form-input" value="{{ api_token }}" readonly>
-                    </div>
+          </div>
-                </div>
+        </div>
-            </div>
+      </div>
-            <p>
+      <p>
-              <strong>Please treat this token as you would any other credential.</strong>
+        <strong>Please treat this token as you would any other credential.</strong>
-              Any party with access to this token can access and manage all your bookmarks.
+        Any party with access to this token can access and manage all your bookmarks.
-              If you think that a token was compromised you can revoke (delete) it in the <a href="{% url 'admin:authtoken_tokenproxy_changelist' %}">admin panel</a>.
+        If you think that a token was compromised you can revoke (delete) it in the <a
-              After deleting the token, a new one will be generated when you reload this settings page.
+          href="{% url 'admin:authtoken_tokenproxy_changelist' %}">admin panel</a>.
-            </p>
+        After deleting the token, a new one will be generated when you reload this settings page.
-        </section>
+      </p>
+    </section>
 
-        <section class="content-area">
+    <section class="content-area">
-            <h2>RSS Feeds</h2>
+      <h2>RSS Feeds</h2>
-            <p>The following URLs provide RSS feeds for your bookmarks:</p>
+      <p>The following URLs provide RSS feeds for your bookmarks:</p>
-            <ul style="list-style-position: outside;">
+      <ul style="list-style-position: outside;">
-              <li><a href="{{ all_feed_url }}">All bookmarks</a></li>
+        <li><a href="{{ all_feed_url }}">All bookmarks</a></li>
-              <li><a href="{{ unread_feed_url }}">Unread bookmarks</a></li>
+        <li><a href="{{ unread_feed_url }}">Unread bookmarks</a></li>
-              <li><a href="{{ shared_feed_url }}">Shared bookmarks</a></li>
+        <li><a href="{{ shared_feed_url }}">Shared bookmarks</a></li>
-              <li><a href="{{ public_shared_feed_url }}">Public shared bookmarks</a><br><span class="text-small text-gray">The public shared feed does not contain an authentication token and can be shared with other people. Only shows shared bookmarks from users who have explicitly enabled public sharing.</span></li>
+        <li><a href="{{ public_shared_feed_url }}">Public shared bookmarks</a><br><span class="text-small text-gray">The public shared feed does not contain an authentication token and can be shared with other people. Only shows shared bookmarks from users who have explicitly enabled public sharing.</span>
-            </ul>
+        </li>
-            <p>
+      </ul>
-              All URLs support appending a <code>q</code> URL parameter for specifying a search query.
+      <p>
-              You can get an example by doing a search in the bookmarks view and then copying the parameter from the URL.
+        All URLs support the following URL parameters:
-            </p>
+      </p>
-            <p>
+      <ul style="list-style-position: outside;">
-              <strong>Please note that these URLs include an authentication token that should be treated like any other credential.</strong>
+        <li>A <code>limit</code> parameter for specifying the maximum number of bookmarks to include in the feed. By
-              Any party with access to these URLs can read all your bookmarks.
+          default, only the latest 100 matching bookmarks are included.
-              If you think that a URL was compromised you can delete the feed token for your user in the <a href="{% url 'admin:bookmarks_feedtoken_changelist' %}">admin panel</a>.
+        </li>
-              After deleting the feed token, new URLs will be generated when you reload this settings page.
+        <li>A <code>q</code> URL parameter for specifying a search query. You can get an example by doing a search in
-            </p>
+          the bookmarks view and then copying the parameter from the URL.
-        </section>
+        </li>
-    </div>
+        <li>An <code>unread</code> parameter for filtering for unread or read bookmarks. Use <code>yes</code> for unread
+          bookmarks and <code>no</code> for read bookmarks.
+        </li>
+        <li>A <code>shared</code> parameter for filtering for shared or unshared bookmarks. Use <code>yes</code> for
+          shared bookmarks and <code>no</code> for unshared bookmarks.
+        </li>
+      </ul>
+      <p>
+        <strong>Please note that these URLs include an authentication token that should be treated like any other
+          credential.</strong>
+        Any party with access to these URLs can read all your bookmarks.
+        If you think that a URL was compromised you can delete the feed token for your user in the <a
+          href="{% url 'admin:bookmarks_feedtoken_changelist' %}">admin panel</a>.
+        After deleting the feed token, new URLs will be generated when you reload this settings page.
+      </p>
+    </section>
+  </div>
 {% endblock %}

bookmarks/tests/helpers.py

@@ -1,6 +1,6 @@
 import random
 import logging
-import datetime
+from datetime import datetime
 from typing import List
 
 from bs4 import BeautifulSoup
@@ -10,7 +10,7 @@ from django.utils.crypto import get_random_string
 from rest_framework import status
 from rest_framework.test import APITestCase
 
-from bookmarks.models import Bookmark, Tag
+from bookmarks.models import Bookmark, BookmarkAsset, Tag
 
 
 class BookmarkFactoryMixin:
@@ -24,6 +24,11 @@ class BookmarkFactoryMixin:
 
         return self.user
 
+    def setup_superuser(self):
+        return User.objects.create_superuser(
+            "superuser", "superuser@example.com", "password123"
+        )
+
     def setup_bookmark(
         self,
         is_archived: bool = False,
@@ -39,6 +44,7 @@ class BookmarkFactoryMixin:
         website_description: str = "",
         web_archive_snapshot_url: str = "",
         favicon_file: str = "",
+        preview_image_file: str = "",
         added: datetime = None,
     ):
         if title is None:
@@ -67,6 +73,7 @@ class BookmarkFactoryMixin:
             shared=shared,
             web_archive_snapshot_url=web_archive_snapshot_url,
             favicon_file=favicon_file,
+            preview_image_file=preview_image_file,
         )
         bookmark.save()
         for tag in tags:
@@ -85,6 +92,8 @@ class BookmarkFactoryMixin:
         shared: bool = False,
         with_tags: bool = False,
         with_web_archive_snapshot_url: bool = False,
+        with_favicon_file: bool = False,
+        with_preview_image_file: bool = False,
         user: User = None,
     ):
         user = user or self.get_or_create_test_user()
@@ -116,6 +125,12 @@ class BookmarkFactoryMixin:
             web_archive_snapshot_url = ""
             if with_web_archive_snapshot_url:
                 web_archive_snapshot_url = f"https://web.archive.org/web/{i}"
+            favicon_file = ""
+            if with_favicon_file:
+                favicon_file = f"favicon_{i}.png"
+            preview_image_file = ""
+            if with_preview_image_file:
+                preview_image_file = f"preview_image_{i}.png"
             bookmark = self.setup_bookmark(
                 url=url,
                 title=title,
@@ -124,6 +139,8 @@ class BookmarkFactoryMixin:
                 shared=shared,
                 tags=tags,
                 web_archive_snapshot_url=web_archive_snapshot_url,
+                favicon_file=favicon_file,
+                preview_image_file=preview_image_file,
                 user=user,
             )
             bookmarks.append(bookmark)
@@ -133,6 +150,38 @@ class BookmarkFactoryMixin:
     def get_numbered_bookmark(self, title: str):
         return Bookmark.objects.get(title=title)
 
+    def setup_asset(
+        self,
+        bookmark: Bookmark,
+        date_created: datetime = None,
+        file: str = None,
+        file_size: int = None,
+        asset_type: str = BookmarkAsset.TYPE_SNAPSHOT,
+        content_type: str = "image/html",
+        display_name: str = None,
+        status: str = BookmarkAsset.STATUS_COMPLETE,
+        gzip: bool = False,
+    ):
+        if date_created is None:
+            date_created = timezone.now()
+        if not file:
+            file = get_random_string(length=32)
+        if not display_name:
+            display_name = file
+        asset = BookmarkAsset(
+            bookmark=bookmark,
+            date_created=date_created,
+            file=file,
+            file_size=file_size,
+            asset_type=asset_type,
+            content_type=content_type,
+            display_name=display_name,
+            status=status,
+            gzip=gzip,
+        )
+        asset.save()
+        return asset
+
     def setup_tag(self, user: User = None, name: str = ""):
         if user is None:
             user = self.get_or_create_test_user()

bookmarks/tests/test_anonymous_view.py

@@ -1,29 +0,0 @@
-from django.test import TestCase
-from django.urls import reverse
-
-from bookmarks.tests.helpers import BookmarkFactoryMixin
-
-
-class AnonymousViewTestCase(TestCase, BookmarkFactoryMixin):
-    def assertSharedBookmarksLinkCount(self, response, count):
-        url = reverse("bookmarks:shared")
-        self.assertContains(
-            response,
-            f'<a href="{url}" class="btn btn-link">Shared bookmarks</a>',
-            count=count,
-        )
-
-    def test_publicly_shared_bookmarks_link(self):
-        # should not render link if no public shares exist
-        user = self.setup_user(enable_sharing=True)
-        self.setup_bookmark(user=user, shared=True)
-
-        response = self.client.get(reverse("login"))
-        self.assertSharedBookmarksLinkCount(response, 0)
-
-        # should render link if public shares exist
-        user.profile.enable_public_sharing = True
-        user.profile.save()
-
-        response = self.client.get(reverse("login"))
-        self.assertSharedBookmarksLinkCount(response, 1)

bookmarks/tests/test_auto_tagging.py

@@ -0,0 +1,190 @@
+from bookmarks.services import auto_tagging
+from django.test import TestCase
+
+
+class AutoTaggingTestCase(TestCase):
+    def test_auto_tag_by_domain(self):
+        script = """
+            example.com example
+            test.com test
+        """
+        url = "https://example.com/"
+
+        tags = auto_tagging.get_tags(script, url)
+
+        self.assertEqual(tags, {"example"})
+
+    def test_auto_tag_by_domain_works_with_port(self):
+        script = """
+            example.com example
+            test.com test
+        """
+        url = "https://example.com:8080/"
+
+        tags = auto_tagging.get_tags(script, url)
+
+        self.assertEqual(tags, {"example"})
+
+    def test_auto_tag_by_domain_ignores_case(self):
+        script = """
+            EXAMPLE.com example
+        """
+        url = "https://example.com/"
+
+        tags = auto_tagging.get_tags(script, url)
+
+        self.assertEqual(tags, {"example"})
+
+    def test_auto_tag_by_domain_should_add_all_tags(self):
+        script = """
+            example.com one two three
+        """
+        url = "https://example.com/"
+
+        tags = auto_tagging.get_tags(script, url)
+
+        self.assertEqual(tags, {"one", "two", "three"})
+
+    def test_auto_tag_by_domain_work_with_idn_domains(self):
+        script = """
+            रजिस्ट्री.भारत tag1
+        """
+        url = "https://www.xn--81bg3cc2b2bk5hb.xn--h2brj9c/"
+
+        tags = auto_tagging.get_tags(script, url)
+
+        self.assertEqual(tags, {"tag1"})
+
+        script = """
+            xn--81bg3cc2b2bk5hb.xn--h2brj9c tag1
+        """
+        url = "https://www.रजिस्ट्री.भारत/"
+
+        tags = auto_tagging.get_tags(script, url)
+
+        self.assertEqual(tags, {"tag1"})
+
+    def test_auto_tag_by_domain_and_path(self):
+        script = """
+            example.com/one one
+            example.com/two two
+            test.com test
+        """
+        url = "https://example.com/one/"
+
+        tags = auto_tagging.get_tags(script, url)
+
+        self.assertEqual(tags, {"one"})
+
+    def test_auto_tag_by_domain_and_path_ignores_case(self):
+        script = """
+            example.com/One one
+        """
+        url = "https://example.com/one/"
+
+        tags = auto_tagging.get_tags(script, url)
+
+        self.assertEqual(tags, {"one"})
+
+    def test_auto_tag_by_domain_and_path_matches_path_ltr(self):
+        script = """
+            example.com/one one
+            example.com/two two
+            test.com test
+        """
+        url = "https://example.com/one/two"
+
+        tags = auto_tagging.get_tags(script, url)
+
+        self.assertEqual(tags, {"one"})
+
+    def test_auto_tag_by_domain_ignores_domain_in_path(self):
+        script = """
+            example.com example
+        """
+        url = "https://test.com/example.com"
+
+        tags = auto_tagging.get_tags(script, url)
+
+        self.assertEqual(tags, set([]))
+
+    def test_auto_tag_by_domain_includes_subdomains(self):
+        script = """
+            example.com example
+            test.example.com test
+            some.example.com some
+        """
+        url = "https://test.example.com/"
+
+        tags = auto_tagging.get_tags(script, url)
+
+        self.assertEqual(tags, {"example", "test"})
+
+    def test_auto_tag_by_domain_matches_domain_rtl(self):
+        script = """
+            example.com example
+        """
+        url = "https://example.com.bad-website.com/"
+
+        tags = auto_tagging.get_tags(script, url)
+
+        self.assertEqual(tags, set([]))
+
+    def test_auto_tag_by_domain_ignores_schema(self):
+        script = """
+            https://example.com/ https
+            http://example.com/ http
+        """
+        url = "http://example.com/"
+
+        tags = auto_tagging.get_tags(script, url)
+
+        self.assertEqual(tags, {"https", "http"})
+
+    def test_auto_tag_by_domain_ignores_lines_with_no_tags(self):
+        script = """
+            example.com
+        """
+        url = "https://example.com/"
+
+        tags = auto_tagging.get_tags(script, url)
+
+        self.assertEqual(tags, set([]))
+
+    def test_auto_tag_by_domain_path_and_qs(self):
+        script = """
+            example.com/page?a=b tag1     # true, matches a=b
+            example.com/page?a=c&c=d tag2 # true, matches both a=c and c=d
+            example.com/page?c=d&l=p tag3 # false, l=p doesn't exists
+            example.com/page?a=bb tag4    # false bb != b
+            example.com/page?a=b&a=c tag5 # true, matches both a=b and a=c
+            example.com/page?a=B tag6     # true, matches a=b because case insensitive
+            example.com/page?A=b tag7     # true, matches a=b because case insensitive
+        """
+        url = "https://example.com/page/some?z=x&a=b&v=b&c=d&o=p&a=c"
+
+        tags = auto_tagging.get_tags(script, url)
+
+        self.assertEqual(tags, {"tag1", "tag2", "tag5", "tag6", "tag7"})
+
+    def test_auto_tag_by_domain_path_and_qs_with_empty_value(self):
+        script = """
+            example.com/page?a= tag1
+            example.com/page?b= tag2
+        """
+        url = "https://example.com/page/some?a=value"
+
+        tags = auto_tagging.get_tags(script, url)
+
+        self.assertEqual(tags, {"tag1"})
+
+    def test_auto_tag_by_domain_path_and_qs_works_with_encoded_url(self):
+        script = """
+            example.com/page?a=йцу tag1
+            example.com/page?a=%D0%B9%D1%86%D1%83 tag2
+        """
+        url = "https://example.com/page?a=%D0%B9%D1%86%D1%83"
+
+        tags = auto_tagging.get_tags(script, url)
+
+        self.assertEqual(tags, {"tag1", "tag2"})

bookmarks/tests/test_bookmark_archived_view.py

@@ -94,15 +94,10 @@ class BookmarkArchivedViewTestCase(TestCase, BookmarkFactoryMixin, HtmlTestMixin
         )
 
     def assertBulkActionForm(self, response, url: str):
-        html = collapse_whitespace(response.content.decode())
+        soup = self.make_soup(response.content.decode())
-        needle = collapse_whitespace(
+        form = soup.select_one("form.bookmark-actions")
-            f"""
+        self.assertIsNotNone(form)
-            <form class="bookmark-actions"
+        self.assertEqual(form.attrs["action"], url)
-                action="{url}"
-                method="post" autocomplete="off">
-        """
-        )
-        self.assertIn(needle, html)
 
     def test_should_list_archived_and_user_owned_bookmarks(self):
         other_user = User.objects.create_user(

bookmarks/tests/test_bookmark_asset_view.py

@@ -0,0 +1,137 @@
+import os
+
+from django.conf import settings
+from django.test import TestCase
+from django.urls import reverse
+
+from bookmarks.tests.helpers import (
+    BookmarkFactoryMixin,
+)
+
+
+class BookmarkAssetViewTestCase(TestCase, BookmarkFactoryMixin):
+    def setUp(self) -> None:
+        user = self.get_or_create_test_user()
+        self.client.force_login(user)
+
+    def tearDown(self):
+        temp_files = [
+            f for f in os.listdir(settings.LD_ASSET_FOLDER) if f.startswith("temp")
+        ]
+        for temp_file in temp_files:
+            os.remove(os.path.join(settings.LD_ASSET_FOLDER, temp_file))
+
+    def setup_asset_file(self, filename):
+        if not os.path.exists(settings.LD_ASSET_FOLDER):
+            os.makedirs(settings.LD_ASSET_FOLDER)
+        filepath = os.path.join(settings.LD_ASSET_FOLDER, filename)
+        with open(filepath, "w") as f:
+            f.write("test")
+
+    def setup_asset_with_file(self, bookmark):
+        filename = f"temp_{bookmark.id}.html.gzip"
+        self.setup_asset_file(filename)
+        asset = self.setup_asset(bookmark=bookmark, file=filename)
+        return asset
+
+    def view_access_test(self, view_name: str):
+        # own bookmark
+        bookmark = self.setup_bookmark()
+        asset = self.setup_asset_with_file(bookmark)
+
+        response = self.client.get(reverse(view_name, args=[asset.id]))
+        self.assertEqual(response.status_code, 200)
+
+        # other user's bookmark
+        other_user = self.setup_user()
+        bookmark = self.setup_bookmark(user=other_user)
+        asset = self.setup_asset_with_file(bookmark)
+
+        response = self.client.get(reverse(view_name, args=[asset.id]))
+        self.assertEqual(response.status_code, 404)
+
+        # shared, sharing disabled
+        bookmark = self.setup_bookmark(user=other_user, shared=True)
+        asset = self.setup_asset_with_file(bookmark)
+
+        response = self.client.get(reverse(view_name, args=[asset.id]))
+        self.assertEqual(response.status_code, 404)
+
+        # unshared, sharing enabled
+        profile = other_user.profile
+        profile.enable_sharing = True
+        profile.save()
+        bookmark = self.setup_bookmark(user=other_user, shared=False)
+        asset = self.setup_asset_with_file(bookmark)
+
+        response = self.client.get(reverse(view_name, args=[asset.id]))
+        self.assertEqual(response.status_code, 404)
+
+        # shared, sharing enabled
+        bookmark = self.setup_bookmark(user=other_user, shared=True)
+        asset = self.setup_asset_with_file(bookmark)
+
+        response = self.client.get(reverse(view_name, args=[asset.id]))
+        self.assertEqual(response.status_code, 200)
+
+    def view_access_guest_user_test(self, view_name: str):
+        self.client.logout()
+
+        # unshared, sharing disabled
+        bookmark = self.setup_bookmark()
+        asset = self.setup_asset_with_file(bookmark)
+
+        response = self.client.get(reverse(view_name, args=[asset.id]))
+        self.assertEqual(response.status_code, 404)
+
+        # shared, sharing disabled
+        bookmark = self.setup_bookmark(shared=True)
+        asset = self.setup_asset_with_file(bookmark)
+
+        response = self.client.get(reverse(view_name, args=[asset.id]))
+        self.assertEqual(response.status_code, 404)
+
+        # unshared, sharing enabled
+        profile = self.get_or_create_test_user().profile
+        profile.enable_sharing = True
+        profile.save()
+        bookmark = self.setup_bookmark(shared=False)
+        asset = self.setup_asset_with_file(bookmark)
+
+        response = self.client.get(reverse(view_name, args=[asset.id]))
+        self.assertEqual(response.status_code, 404)
+
+        # shared, sharing enabled
+        bookmark = self.setup_bookmark(shared=True)
+        asset = self.setup_asset_with_file(bookmark)
+
+        response = self.client.get(reverse(view_name, args=[asset.id]))
+        self.assertEqual(response.status_code, 404)
+
+        # unshared, public sharing enabled
+        profile.enable_public_sharing = True
+        profile.save()
+        bookmark = self.setup_bookmark(shared=False)
+        asset = self.setup_asset_with_file(bookmark)
+
+        response = self.client.get(reverse(view_name, args=[asset.id]))
+        self.assertEqual(response.status_code, 404)
+
+        # shared, public sharing enabled
+        bookmark = self.setup_bookmark(shared=True)
+        asset = self.setup_asset_with_file(bookmark)
+
+        response = self.client.get(reverse(view_name, args=[asset.id]))
+        self.assertEqual(response.status_code, 200)
+
+    def test_view_access(self):
+        self.view_access_test("bookmarks:assets.view")
+
+    def test_view_access_guest_user(self):
+        self.view_access_guest_user_test("bookmarks:assets.view")
+
+    def test_reader_view_access(self):
+        self.view_access_test("bookmarks:assets.read")
+
+    def test_reader_view_access_guest_user(self):
+        self.view_access_guest_user_test("bookmarks:assets.read")

bookmarks/tests/test_bookmark_assets.py

@@ -0,0 +1,89 @@
+import os
+
+from django.conf import settings
+from django.test import TestCase
+
+from bookmarks.tests.helpers import (
+    BookmarkFactoryMixin,
+)
+from bookmarks.services import bookmarks
+
+
+class BookmarkAssetsTestCase(TestCase, BookmarkFactoryMixin):
+    def tearDown(self):
+        temp_files = [
+            f for f in os.listdir(settings.LD_ASSET_FOLDER) if f.startswith("temp")
+        ]
+        for temp_file in temp_files:
+            os.remove(os.path.join(settings.LD_ASSET_FOLDER, temp_file))
+
+    def setup_asset_file(self, filename):
+        if not os.path.exists(settings.LD_ASSET_FOLDER):
+            os.makedirs(settings.LD_ASSET_FOLDER)
+        filepath = os.path.join(settings.LD_ASSET_FOLDER, filename)
+        with open(filepath, "w") as f:
+            f.write("test")
+
+    def setup_asset_with_file(self, bookmark):
+        filename = f"temp_{bookmark.id}.html.gzip"
+        self.setup_asset_file(filename)
+        asset = self.setup_asset(bookmark=bookmark, file=filename)
+        return asset
+
+    def test_delete_bookmark_deletes_asset_file(self):
+        bookmark = self.setup_bookmark()
+        asset = self.setup_asset_with_file(bookmark)
+        self.assertTrue(
+            os.path.exists(os.path.join(settings.LD_ASSET_FOLDER, asset.file))
+        )
+
+        bookmark.delete()
+        self.assertFalse(
+            os.path.exists(os.path.join(settings.LD_ASSET_FOLDER, asset.file))
+        )
+
+    def test_bulk_delete_bookmarks_deletes_asset_files(self):
+        bookmark1 = self.setup_bookmark()
+        asset1 = self.setup_asset_with_file(bookmark1)
+        bookmark2 = self.setup_bookmark()
+        asset2 = self.setup_asset_with_file(bookmark2)
+        bookmark3 = self.setup_bookmark()
+        asset3 = self.setup_asset_with_file(bookmark3)
+        self.assertTrue(
+            os.path.exists(os.path.join(settings.LD_ASSET_FOLDER, asset1.file))
+        )
+        self.assertTrue(
+            os.path.exists(os.path.join(settings.LD_ASSET_FOLDER, asset2.file))
+        )
+        self.assertTrue(
+            os.path.exists(os.path.join(settings.LD_ASSET_FOLDER, asset3.file))
+        )
+
+        bookmarks.delete_bookmarks(
+            [bookmark1.id, bookmark2.id, bookmark3.id], self.get_or_create_test_user()
+        )
+
+        self.assertFalse(
+            os.path.exists(os.path.join(settings.LD_ASSET_FOLDER, asset1.file))
+        )
+        self.assertFalse(
+            os.path.exists(os.path.join(settings.LD_ASSET_FOLDER, asset2.file))
+        )
+        self.assertFalse(
+            os.path.exists(os.path.join(settings.LD_ASSET_FOLDER, asset3.file))
+        )
+
+    def test_save_updates_file_size(self):
+        # File does not exist initially
+        bookmark = self.setup_bookmark()
+        asset = self.setup_asset(bookmark=bookmark, file="temp.html.gz")
+        self.assertIsNone(asset.file_size)
+
+        # Add file, save again
+        self.setup_asset_file(asset.file)
+        asset.save()
+        self.assertEqual(asset.file_size, 4)
+
+        # Create asset with initial file
+        asset = self.setup_asset(bookmark=bookmark, file="temp.html.gz")
+        self.assertEqual(asset.file_size, 4)

bookmarks/tests/test_bookmark_details_modal.py

@@ -1,8 +1,14 @@
-from django.test import TestCase
+import datetime
-from django.urls import reverse
+import re
-from django.utils import formats
+from unittest.mock import patch
 
-from bookmarks.models import UserProfile
+from django.core.files.uploadedfile import SimpleUploadedFile
+from django.test import TestCase, override_settings
+from django.urls import reverse
+from django.utils import formats, timezone
+
+from bookmarks.models import BookmarkAsset, UserProfile
+from bookmarks.services import bookmarks, tasks
 from bookmarks.tests.helpers import BookmarkFactoryMixin, HtmlTestMixin
 
 
@@ -11,8 +17,15 @@ class BookmarkDetailsModalTestCase(TestCase, BookmarkFactoryMixin, HtmlTestMixin
         user = self.get_or_create_test_user()
         self.client.force_login(user)
 
+    def get_view_name(self):
+        return "bookmarks:details_modal"
+
     def get_base_url(self, bookmark):
-        return reverse("bookmarks:details_modal", args=[bookmark.id])
+        return reverse(self.get_view_name(), args=[bookmark.id])
+
+    def get_details_form(self, soup, bookmark):
+        expected_url = reverse("bookmarks:details", args=[bookmark.id])
+        return soup.find("form", {"action": expected_url})
 
     def get_details(self, bookmark, return_url=""):
         url = self.get_base_url(bookmark)
@@ -35,43 +48,41 @@ class BookmarkDetailsModalTestCase(TestCase, BookmarkFactoryMixin, HtmlTestMixin
     def find_weblink(self, soup, url):
         return soup.find("a", {"class": "weblink", "href": url})
 
-    def test_access(self):
+    def count_weblinks(self, soup):
+        return len(soup.find_all("a", {"class": "weblink"}))
+
+    def find_asset(self, soup, asset):
+        return soup.find("div", {"data-asset-id": asset.id})
+
+    def details_route_access_test(self, view_name: str, shareable: bool):
         # own bookmark
         bookmark = self.setup_bookmark()
 
-        response = self.client.get(
+        response = self.client.get(reverse(view_name, args=[bookmark.id]))
-            reverse("bookmarks:details_modal", args=[bookmark.id])
-        )
         self.assertEqual(response.status_code, 200)
 
         # other user's bookmark
         other_user = self.setup_user()
         bookmark = self.setup_bookmark(user=other_user)
 
-        response = self.client.get(
+        response = self.client.get(reverse(view_name, args=[bookmark.id]))
-            reverse("bookmarks:details_modal", args=[bookmark.id])
-        )
         self.assertEqual(response.status_code, 404)
 
         # non-existent bookmark
-        response = self.client.get(reverse("bookmarks:details_modal", args=[9999]))
+        response = self.client.get(reverse(view_name, args=[9999]))
         self.assertEqual(response.status_code, 404)
 
         # guest user
         self.client.logout()
-        response = self.client.get(
+        response = self.client.get(reverse(view_name, args=[bookmark.id]))
-            reverse("bookmarks:details_modal", args=[bookmark.id])
+        self.assertEqual(response.status_code, 404 if shareable else 302)
-        )
-        self.assertEqual(response.status_code, 404)
 
-    def test_access_with_sharing(self):
+    def details_route_sharing_access_test(self, view_name: str, shareable: bool):
         # shared bookmark, sharing disabled
         other_user = self.setup_user()
         bookmark = self.setup_bookmark(shared=True, user=other_user)
 
-        response = self.client.get(
+        response = self.client.get(reverse(view_name, args=[bookmark.id]))
-            reverse("bookmarks:details_modal", args=[bookmark.id])
-        )
         self.assertEqual(response.status_code, 404)
 
         # shared bookmark, sharing enabled
@@ -79,26 +90,32 @@ class BookmarkDetailsModalTestCase(TestCase, BookmarkFactoryMixin, HtmlTestMixin
         profile.enable_sharing = True
         profile.save()
 
-        response = self.client.get(
+        response = self.client.get(reverse(view_name, args=[bookmark.id]))
-            reverse("bookmarks:details_modal", args=[bookmark.id])
+        self.assertEqual(response.status_code, 200 if shareable else 404)
-        )
-        self.assertEqual(response.status_code, 200)
 
         # shared bookmark, guest user, no public sharing
         self.client.logout()
-        response = self.client.get(
+        response = self.client.get(reverse(view_name, args=[bookmark.id]))
-            reverse("bookmarks:details_modal", args=[bookmark.id])
+        self.assertEqual(response.status_code, 404 if shareable else 302)
-        )
-        self.assertEqual(response.status_code, 404)
 
         # shared bookmark, guest user, public sharing
         profile.enable_public_sharing = True
         profile.save()
 
-        response = self.client.get(
+        response = self.client.get(reverse(view_name, args=[bookmark.id]))
-            reverse("bookmarks:details_modal", args=[bookmark.id])
+        self.assertEqual(response.status_code, 200 if shareable else 302)
-        )
+
-        self.assertEqual(response.status_code, 200)
+    def test_access(self):
+        self.details_route_access_test(self.get_view_name(), True)
+
+    def test_access_with_sharing(self):
+        self.details_route_sharing_access_test(self.get_view_name(), True)
+
+    def test_assets_access(self):
+        self.details_route_access_test("bookmarks:details_assets", True)
+
+    def test_assets_access_with_sharing(self):
+        self.details_route_sharing_access_test("bookmarks:details_assets", True)
 
     def test_displays_title(self):
         # with title
@@ -160,20 +177,55 @@ class BookmarkDetailsModalTestCase(TestCase, BookmarkFactoryMixin, HtmlTestMixin
         self.assertIsNotNone(image)
         self.assertEqual(image["src"], "/static/example.png")
 
-    def test_internet_archive_link(self):
+    def test_reader_mode_link(self):
-        # without snapshot url
+        # no latest snapshot
         bookmark = self.setup_bookmark()
         soup = self.get_details(bookmark)
-        link = self.find_weblink(soup, bookmark.web_archive_snapshot_url)
+        self.assertEqual(self.count_weblinks(soup), 2)
-        self.assertIsNone(link)
 
-        # with snapshot url
+        # snapshot is not complete
+        self.setup_asset(
+            bookmark,
+            asset_type=BookmarkAsset.TYPE_SNAPSHOT,
+            status=BookmarkAsset.STATUS_PENDING,
+        )
+        self.setup_asset(
+            bookmark,
+            asset_type=BookmarkAsset.TYPE_SNAPSHOT,
+            status=BookmarkAsset.STATUS_FAILURE,
+        )
+        soup = self.get_details(bookmark)
+        self.assertEqual(self.count_weblinks(soup), 2)
+
+        # not a snapshot
+        self.setup_asset(
+            bookmark,
+            asset_type="upload",
+            status=BookmarkAsset.STATUS_COMPLETE,
+        )
+        soup = self.get_details(bookmark)
+        self.assertEqual(self.count_weblinks(soup), 2)
+
+        # snapshot is complete
+        asset = self.setup_asset(
+            bookmark,
+            asset_type=BookmarkAsset.TYPE_SNAPSHOT,
+            status=BookmarkAsset.STATUS_COMPLETE,
+        )
+        soup = self.get_details(bookmark)
+        self.assertEqual(self.count_weblinks(soup), 3)
+
+        reader_mode_url = reverse("bookmarks:assets.read", args=[asset.id])
+        link = self.find_weblink(soup, reader_mode_url)
+        self.assertIsNotNone(link)
+
+    def test_internet_archive_link_with_snapshot_url(self):
         bookmark = self.setup_bookmark(web_archive_snapshot_url="https://example.com/")
         soup = self.get_details(bookmark)
         link = self.find_weblink(soup, bookmark.web_archive_snapshot_url)
         self.assertIsNotNone(link)
         self.assertEqual(link["href"], bookmark.web_archive_snapshot_url)
-        self.assertEqual(link.text.strip(), "View on Internet Archive")
+        self.assertEqual(link.text.strip(), "Internet Archive")
 
         # favicons disabled
         bookmark = self.setup_bookmark(
@@ -206,6 +258,21 @@ class BookmarkDetailsModalTestCase(TestCase, BookmarkFactoryMixin, HtmlTestMixin
         image = link.select_one("svg")
         self.assertIsNotNone(image)
 
+    def test_internet_archive_link_with_fallback_url(self):
+        date_added = timezone.datetime(
+            2023, 8, 11, 21, 45, 11, tzinfo=datetime.timezone.utc
+        )
+        bookmark = self.setup_bookmark(url="https://example.com/", added=date_added)
+        fallback_web_archive_url = (
+            "https://web.archive.org/web/20230811214511/https://example.com/"
+        )
+
+        soup = self.get_details(bookmark)
+        link = self.find_weblink(soup, fallback_web_archive_url)
+        self.assertIsNotNone(link)
+        self.assertEqual(link["href"], fallback_web_archive_url)
+        self.assertEqual(link.text.strip(), "Internet Archive")
+
     def test_weblinks_respect_target_setting(self):
         bookmark = self.setup_bookmark(web_archive_snapshot_url="https://example.com/")
 
@@ -242,13 +309,42 @@ class BookmarkDetailsModalTestCase(TestCase, BookmarkFactoryMixin, HtmlTestMixin
             web_archive_link["target"], UserProfile.BOOKMARK_LINK_TARGET_SELF
         )
 
+    def test_preview_image(self):
+        # without image
+        bookmark = self.setup_bookmark()
+        soup = self.get_details(bookmark)
+        image = soup.select_one("div.preview-image img")
+        self.assertIsNone(image)
+
+        # with image
+        bookmark = self.setup_bookmark(preview_image_file="example.png")
+        soup = self.get_details(bookmark)
+        image = soup.select_one("div.preview-image img")
+        self.assertIsNone(image)
+
+        # preview images enabled, no image
+        profile = self.get_or_create_test_user().profile
+        profile.enable_preview_images = True
+        profile.save()
+
+        bookmark = self.setup_bookmark()
+        soup = self.get_details(bookmark)
+        image = soup.select_one("div.preview-image img")
+        self.assertIsNone(image)
+
+        # preview images enabled, image present
+        bookmark = self.setup_bookmark(preview_image_file="example.png")
+        soup = self.get_details(bookmark)
+        image = soup.select_one("div.preview-image img")
+        self.assertIsNotNone(image)
+        self.assertEqual(image["src"], "/static/example.png")
+
     def test_status(self):
         # renders form
         bookmark = self.setup_bookmark()
         soup = self.get_details(bookmark)
-        section = self.get_section(soup, "Status")
 
-        form = section.find("form")
+        form = self.get_details_form(soup, bookmark)
         self.assertIsNotNone(form)
         self.assertEqual(
             form["action"], reverse("bookmarks:details", args=[bookmark.id])
@@ -312,30 +408,21 @@ class BookmarkDetailsModalTestCase(TestCase, BookmarkFactoryMixin, HtmlTestMixin
         bookmark = self.setup_bookmark()
         soup = self.get_details(bookmark)
         section = self.find_section(soup, "Status")
-        form_action = reverse("bookmarks:details", args=[bookmark.id])
-        form = soup.find("form", {"action": form_action})
         self.assertIsNotNone(section)
-        self.assertIsNotNone(form)
 
         # other user's bookmark
         other_user = self.setup_user(enable_sharing=True)
         bookmark = self.setup_bookmark(user=other_user, shared=True)
         soup = self.get_details(bookmark)
         section = self.find_section(soup, "Status")
-        form_action = reverse("bookmarks:details", args=[bookmark.id])
-        form = soup.find("form", {"action": form_action})
         self.assertIsNone(section)
-        self.assertIsNone(form)
 
         # guest user
         self.client.logout()
         bookmark = self.setup_bookmark(user=other_user, shared=True)
         soup = self.get_details(bookmark)
         section = self.find_section(soup, "Status")
-        form_action = reverse("bookmarks:details", args=[bookmark.id])
-        form = soup.find("form", {"action": form_action})
         self.assertIsNone(section)
-        self.assertIsNone(form)
 
     def test_status_update(self):
         bookmark = self.setup_bookmark()
@@ -560,3 +647,289 @@ class BookmarkDetailsModalTestCase(TestCase, BookmarkFactoryMixin, HtmlTestMixin
         delete_button = soup.find("button", {"type": "submit", "name": "remove"})
         self.assertIsNone(edit_link)
         self.assertIsNone(delete_button)
+
+    def test_assets_visibility_no_snapshot_support(self):
+        bookmark = self.setup_bookmark()
+
+        soup = self.get_details(bookmark)
+        section = self.find_section(soup, "Files")
+        self.assertIsNone(section)
+
+    @override_settings(LD_ENABLE_SNAPSHOTS=True)
+    def test_assets_visibility_with_snapshot_support(self):
+        bookmark = self.setup_bookmark()
+
+        soup = self.get_details(bookmark)
+        section = self.find_section(soup, "Files")
+        self.assertIsNotNone(section)
+
+    @override_settings(LD_ENABLE_SNAPSHOTS=True)
+    def test_asset_list_visibility(self):
+        # no assets
+        bookmark = self.setup_bookmark()
+
+        soup = self.get_details(bookmark)
+        section = self.get_section(soup, "Files")
+        asset_list = section.find("div", {"class": "assets"})
+        self.assertIsNone(asset_list)
+
+        # with assets
+        bookmark = self.setup_bookmark()
+        self.setup_asset(bookmark)
+
+        soup = self.get_details(bookmark)
+        section = self.get_section(soup, "Files")
+        asset_list = section.find("div", {"class": "assets"})
+        self.assertIsNotNone(asset_list)
+
+    @override_settings(LD_ENABLE_SNAPSHOTS=True)
+    def test_asset_list(self):
+        bookmark = self.setup_bookmark()
+        assets = [
+            self.setup_asset(bookmark),
+            self.setup_asset(bookmark),
+            self.setup_asset(bookmark),
+        ]
+
+        soup = self.get_details(bookmark)
+        section = self.get_section(soup, "Files")
+        asset_list = section.find("div", {"class": "assets"})
+
+        for asset in assets:
+            asset_item = self.find_asset(asset_list, asset)
+            self.assertIsNotNone(asset_item)
+
+            asset_icon = asset_item.select_one(".asset-icon svg")
+            self.assertIsNotNone(asset_icon)
+
+            asset_text = asset_item.select_one(".asset-text span")
+            self.assertIsNotNone(asset_text)
+            self.assertIn(asset.display_name, asset_text.text)
+
+            view_url = reverse("bookmarks:assets.view", args=[asset.id])
+            view_link = asset_item.find("a", {"href": view_url})
+            self.assertIsNotNone(view_link)
+
+    @override_settings(LD_ENABLE_SNAPSHOTS=True)
+    def test_asset_without_file(self):
+        bookmark = self.setup_bookmark()
+        asset = self.setup_asset(bookmark)
+        asset.file = ""
+        asset.save()
+
+        soup = self.get_details(bookmark)
+        asset_item = self.find_asset(soup, asset)
+        view_url = reverse("bookmarks:assets.view", args=[asset.id])
+        view_link = asset_item.find("a", {"href": view_url})
+        self.assertIsNone(view_link)
+
+    @override_settings(LD_ENABLE_SNAPSHOTS=True)
+    def test_asset_status(self):
+        bookmark = self.setup_bookmark()
+        pending_asset = self.setup_asset(bookmark, status=BookmarkAsset.STATUS_PENDING)
+        failed_asset = self.setup_asset(bookmark, status=BookmarkAsset.STATUS_FAILURE)
+
+        soup = self.get_details(bookmark)
+
+        asset_item = self.find_asset(soup, pending_asset)
+        asset_text = asset_item.select_one(".asset-text span")
+        self.assertIn("(queued)", asset_text.text)
+
+        asset_item = self.find_asset(soup, failed_asset)
+        asset_text = asset_item.select_one(".asset-text span")
+        self.assertIn("(failed)", asset_text.text)
+
+    @override_settings(LD_ENABLE_SNAPSHOTS=True)
+    def test_asset_file_size(self):
+        bookmark = self.setup_bookmark()
+        asset1 = self.setup_asset(bookmark, file_size=None)
+        asset2 = self.setup_asset(bookmark, file_size=54639)
+        asset3 = self.setup_asset(bookmark, file_size=11492020)
+
+        soup = self.get_details(bookmark)
+
+        asset_item = self.find_asset(soup, asset1)
+        asset_text = asset_item.select_one(".asset-text")
+        self.assertEqual(asset_text.text.strip(), asset1.display_name)
+
+        asset_item = self.find_asset(soup, asset2)
+        asset_text = asset_item.select_one(".asset-text")
+        self.assertIn("53.4\xa0KB", asset_text.text)
+
+        asset_item = self.find_asset(soup, asset3)
+        asset_text = asset_item.select_one(".asset-text")
+        self.assertIn("11.0\xa0MB", asset_text.text)
+
+    @override_settings(LD_ENABLE_SNAPSHOTS=True)
+    def test_asset_actions_visibility(self):
+        bookmark = self.setup_bookmark()
+
+        # with file
+        asset = self.setup_asset(bookmark)
+        soup = self.get_details(bookmark)
+
+        asset_item = self.find_asset(soup, asset)
+        view_link = asset_item.find("a", string="View")
+        delete_button = asset_item.find(
+            "button", {"type": "submit", "name": "remove_asset"}
+        )
+        self.assertIsNotNone(view_link)
+        self.assertIsNotNone(delete_button)
+
+        # without file
+        asset.file = ""
+        asset.save()
+        soup = self.get_details(bookmark)
+
+        asset_item = self.find_asset(soup, asset)
+        view_link = asset_item.find("a", string="View")
+        delete_button = asset_item.find(
+            "button", {"type": "submit", "name": "remove_asset"}
+        )
+        self.assertIsNone(view_link)
+        self.assertIsNotNone(delete_button)
+
+        # shared bookmark
+        other_user = self.setup_user(enable_sharing=True, enable_public_sharing=True)
+        bookmark = self.setup_bookmark(shared=True, user=other_user)
+        asset = self.setup_asset(bookmark)
+        soup = self.get_details(bookmark)
+
+        asset_item = self.find_asset(soup, asset)
+        view_link = asset_item.find("a", string="View")
+        delete_button = asset_item.find(
+            "button", {"type": "submit", "name": "remove_asset"}
+        )
+        self.assertIsNotNone(view_link)
+        self.assertIsNone(delete_button)
+
+        # shared bookmark, guest user
+        self.client.logout()
+        soup = self.get_details(bookmark)
+
+        asset_item = self.find_asset(soup, asset)
+        view_link = asset_item.find("a", string="View")
+        delete_button = asset_item.find(
+            "button", {"type": "submit", "name": "remove_asset"}
+        )
+        self.assertIsNotNone(view_link)
+        self.assertIsNone(delete_button)
+
+    def test_remove_asset(self):
+        # remove asset
+        bookmark = self.setup_bookmark()
+        asset = self.setup_asset(bookmark)
+
+        response = self.client.post(
+            self.get_base_url(bookmark), {"remove_asset": asset.id}
+        )
+        self.assertEqual(response.status_code, 302)
+        self.assertFalse(BookmarkAsset.objects.filter(id=asset.id).exists())
+
+        # non-existent asset
+        response = self.client.post(self.get_base_url(bookmark), {"remove_asset": 9999})
+        self.assertEqual(response.status_code, 404)
+
+        # post without asset ID does not remove
+        asset = self.setup_asset(bookmark)
+        response = self.client.post(self.get_base_url(bookmark))
+        self.assertEqual(response.status_code, 302)
+        self.assertTrue(BookmarkAsset.objects.filter(id=asset.id).exists())
+
+        # guest user
+        asset = self.setup_asset(bookmark)
+        self.client.logout()
+        response = self.client.post(
+            self.get_base_url(bookmark), {"remove_asset": asset.id}
+        )
+        self.assertEqual(response.status_code, 404)
+        self.assertTrue(BookmarkAsset.objects.filter(id=asset.id).exists())
+
+    @override_settings(LD_ENABLE_SNAPSHOTS=True)
+    def test_assets_refresh_when_having_pending_asset(self):
+        bookmark = self.setup_bookmark()
+        asset = self.setup_asset(bookmark, status=BookmarkAsset.STATUS_COMPLETE)
+        fetch_url = reverse("bookmarks:details_assets", args=[bookmark.id])
+
+        # no pending asset
+        soup = self.get_details(bookmark)
+        files_section = self.find_section(soup, "Files")
+        assets_wrapper = files_section.find("div", {"ld-fetch": fetch_url})
+        self.assertIsNone(assets_wrapper)
+
+        # with pending asset
+        asset.status = BookmarkAsset.STATUS_PENDING
+        asset.save()
+
+        soup = self.get_details(bookmark)
+        files_section = self.find_section(soup, "Files")
+        assets_wrapper = files_section.find("div", {"ld-fetch": fetch_url})
+        self.assertIsNotNone(assets_wrapper)
+
+    @override_settings(LD_ENABLE_SNAPSHOTS=True)
+    def test_create_snapshot(self):
+        with patch.object(
+            tasks, "_create_html_snapshot_task"
+        ) as mock_create_html_snapshot_task:
+            bookmark = self.setup_bookmark()
+            response = self.client.post(
+                self.get_base_url(bookmark), {"create_snapshot": ""}
+            )
+            self.assertEqual(response.status_code, 302)
+
+            self.assertEqual(bookmark.bookmarkasset_set.count(), 1)
+
+    @override_settings(LD_ENABLE_SNAPSHOTS=True)
+    def test_create_snapshot_is_disabled_when_having_pending_asset(self):
+        bookmark = self.setup_bookmark()
+        asset = self.setup_asset(bookmark, status=BookmarkAsset.STATUS_COMPLETE)
+
+        # no pending asset
+        soup = self.get_details(bookmark)
+        files_section = self.find_section(soup, "Files")
+        create_button = files_section.find(
+            "button", string=re.compile("Create HTML snapshot")
+        )
+        self.assertFalse(create_button.has_attr("disabled"))
+
+        # with pending asset
+        asset.status = BookmarkAsset.STATUS_PENDING
+        asset.save()
+
+        soup = self.get_details(bookmark)
+        files_section = self.find_section(soup, "Files")
+        create_button = files_section.find(
+            "button", string=re.compile("Create HTML snapshot")
+        )
+        self.assertTrue(create_button.has_attr("disabled"))
+
+    def test_upload_file(self):
+        bookmark = self.setup_bookmark()
+        file_content = b"file content"
+        upload_file = SimpleUploadedFile("test.txt", file_content)
+
+        with patch.object(bookmarks, "upload_asset") as mock_upload_asset:
+            response = self.client.post(
+                self.get_base_url(bookmark),
+                {"upload_asset": "", "upload_asset_file": upload_file},
+            )
+            self.assertEqual(response.status_code, 302)
+
+            mock_upload_asset.assert_called_once()
+
+            args, kwargs = mock_upload_asset.call_args
+            self.assertEqual(args[0], bookmark)
+
+            upload_file = args[1]
+            self.assertEqual(upload_file.name, "test.txt")
+
+    def test_upload_file_without_file(self):
+        bookmark = self.setup_bookmark()
+
+        with patch.object(bookmarks, "upload_asset") as mock_upload_asset:
+            response = self.client.post(
+                self.get_base_url(bookmark),
+                {"upload_asset": ""},
+            )
+            self.assertEqual(response.status_code, 400)
+            mock_upload_asset.assert_not_called()

bookmarks/tests/test_bookmark_details_view.py

@@ -1,8 +1,6 @@
-from django.urls import reverse
-
 from bookmarks.tests.test_bookmark_details_modal import BookmarkDetailsModalTestCase
 
 
 class BookmarkDetailsViewTestCase(BookmarkDetailsModalTestCase):
-    def get_base_url(self, bookmark):
+    def get_view_name(self):
-        return reverse("bookmarks:details", args=[bookmark.id])
+        return "bookmarks:details"

bookmarks/tests/test_bookmark_index_view.py

@@ -6,11 +6,7 @@ from django.test import TestCase
 from django.urls import reverse
 
 from bookmarks.models import Bookmark, BookmarkSearch, Tag, UserProfile
-from bookmarks.tests.helpers import (
+from bookmarks.tests.helpers import BookmarkFactoryMixin, HtmlTestMixin
-    BookmarkFactoryMixin,
-    HtmlTestMixin,
-    collapse_whitespace,
-)
 
 
 class BookmarkIndexViewTestCase(TestCase, BookmarkFactoryMixin, HtmlTestMixin):
@@ -94,15 +90,10 @@ class BookmarkIndexViewTestCase(TestCase, BookmarkFactoryMixin, HtmlTestMixin):
         )
 
     def assertBulkActionForm(self, response, url: str):
-        html = collapse_whitespace(response.content.decode())
+        soup = self.make_soup(response.content.decode())
-        needle = collapse_whitespace(
+        form = soup.select_one("form.bookmark-actions")
-            f"""
+        self.assertIsNotNone(form)
-            <form class="bookmark-actions"
+        self.assertEqual(form.attrs["action"], url)
-                action="{url}"
-                method="post" autocomplete="off">
-        """
-        )
-        self.assertIn(needle, html)
 
     def test_should_list_unarchived_and_user_owned_bookmarks(self):
         other_user = User.objects.create_user(

bookmarks/tests/test_bookmark_new_view.py

@@ -100,6 +100,29 @@ class BookmarkNewViewTestCase(TestCase, BookmarkFactoryMixin):
             html,
         )
 
+    def test_should_prefill_notes_from_url_parameter(self):
+        response = self.client.get(
+            reverse("bookmarks:new")
+            + "?notes=%2A%2AFind%2A%2A%20more%20info%20%5Bhere%5D%28http%3A%2F%2Fexample.com%29"
+        )
+        html = response.content.decode()
+
+        self.assertInHTML(
+            """
+            <details class="notes" open="">
+                <summary>
+                    <span class="form-label d-inline-block">Notes</span>
+                </summary>
+                <label for="id_notes" class="text-assistive">Notes</label>
+                <textarea name="notes" cols="40" rows="8" class="form-input" id="id_notes">**Find** more info [here](http://example.com)</textarea>
+                <div class="form-input-hint">
+                Additional notes, supports Markdown.
+                </div>
+            </details>
+            """,
+            html,
+        )
+
     def test_should_enable_auto_close_when_specified_in_url_parameter(self):
         response = self.client.get(reverse("bookmarks:new") + "?auto_close")
         html = response.content.decode()
@@ -210,3 +233,25 @@ class BookmarkNewViewTestCase(TestCase, BookmarkFactoryMixin):
         response = self.client.get(reverse("bookmarks:edit", args=[bookmark.id]))
 
         self.assertContains(response, '<details class="notes">', count=1)
+
+    def test_should_not_check_unread_by_default(self):
+        response = self.client.get(reverse("bookmarks:new"))
+        html = response.content.decode()
+
+        self.assertInHTML(
+            '<input type="checkbox" name="unread" id="id_unread">',
+            html,
+        )
+
+    def test_should_check_unread_when_configured_in_profile(self):
+        self.user.profile.default_mark_unread = True
+        self.user.profile.save()
+
+        response = self.client.get(reverse("bookmarks:new"))
+        html = response.content.decode()
+
+        self.assertInHTML(
+            '<input type="checkbox" name="unread" value="true" '
+            'id="id_unread" checked="">',
+            html,
+        )

bookmarks/tests/test_bookmarks_api.py

@@ -36,6 +36,16 @@ class BookmarksApiTestCase(LinkdingApiTestCase, BookmarkFactoryMixin):
             expectation["website_title"] = bookmark.website_title
             expectation["website_description"] = bookmark.website_description
             expectation["web_archive_snapshot_url"] = bookmark.web_archive_snapshot_url
+            expectation["favicon_url"] = (
+                f"http://testserver/static/{bookmark.favicon_file}"
+                if bookmark.favicon_file
+                else None
+            )
+            expectation["preview_image_url"] = (
+                f"http://testserver/static/{bookmark.preview_image_file}"
+                if bookmark.preview_image_file
+                else None
+            )
             expectation["is_archived"] = bookmark.is_archived
             expectation["unread"] = bookmark.unread
             expectation["shared"] = bookmark.shared
@@ -65,7 +75,11 @@ class BookmarksApiTestCase(LinkdingApiTestCase, BookmarkFactoryMixin):
     def test_list_bookmarks_with_more_details(self):
         self.authenticate()
         bookmarks = self.setup_numbered_bookmarks(
-            5, with_tags=True, with_web_archive_snapshot_url=True
+            5,
+            with_tags=True,
+            with_web_archive_snapshot_url=True,
+            with_favicon_file=True,
+            with_preview_image_file=True,
         )
 
         response = self.get(
@@ -171,6 +185,23 @@ class BookmarksApiTestCase(LinkdingApiTestCase, BookmarkFactoryMixin):
         )
         self.assertBookmarkListEqual(response.data["results"], archived_bookmarks)
 
+    def test_list_archived_bookmarks_with_more_details(self):
+        self.authenticate()
+        archived_bookmarks = self.setup_numbered_bookmarks(
+            5,
+            archived=True,
+            with_tags=True,
+            with_web_archive_snapshot_url=True,
+            with_favicon_file=True,
+            with_preview_image_file=True,
+        )
+
+        response = self.get(
+            reverse("bookmarks:bookmark-archived"),
+            expected_status_code=status.HTTP_200_OK,
+        )
+        self.assertBookmarkListEqual(response.data["results"], archived_bookmarks)
+
     def test_list_archived_bookmarks_should_filter_by_query(self):
         self.authenticate()
         search_value = self.get_random_string()
@@ -220,6 +251,26 @@ class BookmarksApiTestCase(LinkdingApiTestCase, BookmarkFactoryMixin):
         )
         self.assertBookmarkListEqual(response.data["results"], shared_bookmarks)
 
+    def test_list_shared_bookmarks_with_more_details(self):
+        self.authenticate()
+
+        other_user = self.setup_user(enable_sharing=True)
+        shared_bookmarks = self.setup_numbered_bookmarks(
+            5,
+            shared=True,
+            user=other_user,
+            with_tags=True,
+            with_web_archive_snapshot_url=True,
+            with_favicon_file=True,
+            with_preview_image_file=True,
+        )
+
+        response = self.get(
+            reverse("bookmarks:bookmark-shared"),
+            expected_status_code=status.HTTP_200_OK,
+        )
+        self.assertBookmarkListEqual(response.data["results"], shared_bookmarks)
+
     def test_list_only_publicly_shared_bookmarks_when_not_logged_in(self):
         user1 = self.setup_user(enable_sharing=True, enable_public_sharing=True)
         user2 = self.setup_user(enable_sharing=True)
@@ -440,6 +491,20 @@ class BookmarksApiTestCase(LinkdingApiTestCase, BookmarkFactoryMixin):
         bookmark = Bookmark.objects.get(url=data["url"])
         self.assertFalse(bookmark.shared)
 
+    def test_create_bookmark_should_add_tags_from_auto_tagging(self):
+        tag1 = self.setup_tag()
+        tag2 = self.setup_tag()
+
+        self.authenticate()
+        profile = self.get_or_create_test_user().profile
+        profile.auto_tagging_rules = f"example.com {tag2.name}"
+        profile.save()
+
+        data = {"url": "https://example.com/", "tag_names": [tag1.name]}
+        self.post(reverse("bookmarks:bookmark-list"), data, status.HTTP_201_CREATED)
+        bookmark = Bookmark.objects.get(url=data["url"])
+        self.assertCountEqual(bookmark.tags.all(), [tag1, tag2])
+
     def test_get_bookmark(self):
         self.authenticate()
         bookmark = self.setup_bookmark()
@@ -512,6 +577,22 @@ class BookmarksApiTestCase(LinkdingApiTestCase, BookmarkFactoryMixin):
         updated_bookmark = Bookmark.objects.get(id=bookmark.id)
         self.assertEqual(updated_bookmark.shared, True)
 
+    def test_update_bookmark_adds_tags_from_auto_tagging(self):
+        bookmark = self.setup_bookmark()
+        tag1 = self.setup_tag()
+        tag2 = self.setup_tag()
+
+        self.authenticate()
+        profile = self.get_or_create_test_user().profile
+        profile.auto_tagging_rules = f"example.com {tag2.name}"
+        profile.save()
+
+        data = {"url": "https://example.com/", "tag_names": [tag1.name]}
+        url = reverse("bookmarks:bookmark-detail", args=[bookmark.id])
+        self.put(url, data, expected_status_code=status.HTTP_200_OK)
+        updated_bookmark = Bookmark.objects.get(id=bookmark.id)
+        self.assertCountEqual(updated_bookmark.tags.all(), [tag1, tag2])
+
     def test_patch_bookmark(self):
         self.authenticate()
         bookmark = self.setup_bookmark()
@@ -583,6 +664,22 @@ class BookmarksApiTestCase(LinkdingApiTestCase, BookmarkFactoryMixin):
         self.assertEqual(updated_bookmark.description, bookmark.description)
         self.assertListEqual(updated_bookmark.tag_names, bookmark.tag_names)
 
+    def test_patch_bookmark_adds_tags_from_auto_tagging(self):
+        bookmark = self.setup_bookmark()
+        tag1 = self.setup_tag()
+        tag2 = self.setup_tag()
+
+        self.authenticate()
+        profile = self.get_or_create_test_user().profile
+        profile.auto_tagging_rules = f"example.com {tag2.name}"
+        profile.save()
+
+        data = {"tag_names": [tag1.name]}
+        url = reverse("bookmarks:bookmark-detail", args=[bookmark.id])
+        self.patch(url, data, expected_status_code=status.HTTP_200_OK)
+        updated_bookmark = Bookmark.objects.get(id=bookmark.id)
+        self.assertCountEqual(updated_bookmark.tags.all(), [tag1, tag2])
+
     def test_delete_bookmark(self):
         self.authenticate()
         bookmark = self.setup_bookmark()
@@ -628,7 +725,10 @@ class BookmarksApiTestCase(LinkdingApiTestCase, BookmarkFactoryMixin):
             website_loader, "load_website_metadata"
         ) as mock_load_website_metadata:
             expected_metadata = WebsiteMetadata(
-                "https://example.com", "Scraped metadata", "Scraped description"
+                "https://example.com",
+                "Scraped metadata",
+                "Scraped description",
+                "https://example.com/preview.png",
             )
             mock_load_website_metadata.return_value = expected_metadata
 
@@ -640,9 +740,10 @@ class BookmarksApiTestCase(LinkdingApiTestCase, BookmarkFactoryMixin):
             metadata = response.data["metadata"]
 
             self.assertIsNotNone(metadata)
-            self.assertIsNotNone(expected_metadata.url, metadata["url"])
+            self.assertEqual(expected_metadata.url, metadata["url"])
-            self.assertIsNotNone(expected_metadata.title, metadata["title"])
+            self.assertEqual(expected_metadata.title, metadata["title"])
-            self.assertIsNotNone(expected_metadata.description, metadata["description"])
+            self.assertEqual(expected_metadata.description, metadata["description"])
+            self.assertEqual(expected_metadata.preview_image, metadata["preview_image"])
 
     def test_check_returns_bookmark_if_url_is_bookmarked(self):
         self.authenticate()
@@ -651,6 +752,8 @@ class BookmarksApiTestCase(LinkdingApiTestCase, BookmarkFactoryMixin):
             url="https://example.com",
             title="Example title",
             description="Example description",
+            favicon_file="favicon.png",
+            preview_image_file="preview.png",
         )
 
         url = reverse("bookmarks:bookmark-check")
@@ -665,6 +768,12 @@ class BookmarksApiTestCase(LinkdingApiTestCase, BookmarkFactoryMixin):
         self.assertEqual(bookmark.url, bookmark_data["url"])
         self.assertEqual(bookmark.title, bookmark_data["title"])
         self.assertEqual(bookmark.description, bookmark_data["description"])
+        self.assertEqual(
+            "http://testserver/static/favicon.png", bookmark_data["favicon_url"]
+        )
+        self.assertEqual(
+            "http://testserver/static/preview.png", bookmark_data["preview_image_url"]
+        )
 
     def test_check_returns_existing_metadata_if_url_is_bookmarked(self):
         self.authenticate()
@@ -687,9 +796,38 @@ class BookmarksApiTestCase(LinkdingApiTestCase, BookmarkFactoryMixin):
 
             mock_load_website_metadata.assert_not_called()
             self.assertIsNotNone(metadata)
-            self.assertIsNotNone(bookmark.url, metadata["url"])
+            self.assertEqual(bookmark.url, metadata["url"])
-            self.assertIsNotNone(bookmark.website_title, metadata["title"])
+            self.assertEqual(bookmark.website_title, metadata["title"])
-            self.assertIsNotNone(bookmark.website_description, metadata["description"])
+            self.assertEqual(bookmark.website_description, metadata["description"])
+            self.assertIsNone(metadata["preview_image"])
+
+    def test_check_returns_no_auto_tags_if_none_configured(self):
+        self.authenticate()
+
+        url = reverse("bookmarks:bookmark-check")
+        check_url = urllib.parse.quote_plus("https://example.com")
+        response = self.get(
+            f"{url}?url={check_url}", expected_status_code=status.HTTP_200_OK
+        )
+        auto_tags = response.data["auto_tags"]
+
+        self.assertCountEqual(auto_tags, [])
+
+    def test_check_returns_matching_auto_tags(self):
+        self.authenticate()
+
+        profile = self.get_or_create_test_user().profile
+        profile.auto_tagging_rules = "example.com tag1 tag2"
+        profile.save()
+
+        url = reverse("bookmarks:bookmark-check")
+        check_url = urllib.parse.quote_plus("https://example.com")
+        response = self.get(
+            f"{url}?url={check_url}", expected_status_code=status.HTTP_200_OK
+        )
+        auto_tags = response.data["auto_tags"]
+
+        self.assertCountEqual(auto_tags, ["tag1", "tag2"])
 
     def test_can_only_access_own_bookmarks(self):
         self.authenticate()