Skip to content

Releases: ArchiveBox/ArchiveBox

v0.9.31-rc: New abx-dl runtime, split-out plugin ecosystem, resumable migrations, and isolated snapshot replay

Choose a tag to compare

@pirate pirate released this 18 May 16:51
c5d11c3

Warning

This is a pre-release for the upcoming ArchiveBox v0.9.x line. The latest stable release is still v0.7.4. Please test on a backup first and report regressions before using this for a production collection.

v0.9.x is a large architectural upgrade from v0.8.x: plugin execution has moved out into a standalone plugin ecosystem, archiving runs through the new one-shot abx-dl CLI, and the old pluggy path has been replaced by an event-driven, append-only-log architecture designed to support future browser-extension capture and p2p sync.

⬇️ RC Instructions: use the :dev Docker image with the repo docker-compose.yml
git clone https://github.com/ArchiveBox/ArchiveBox && cd ArchiveBox
ARCHIVEBOX_IMAGE=archivebox/archivebox:dev docker compose up -d

Highlights

  • 🔌 New plugin system split out into abx-plugins
    The extractor/plugin catalog now lives in github.com/ArchiveBox/abx-plugins, with per-plugin config, dependencies, hooks, docs, and install metadata.

  • New one-shot CLI powered by abx-dl
    ArchiveBox now builds on the standalone downloader at github.com/ArchiveBox/abx-dl, so plugin-based archiving can run inside ArchiveBox or independently as a focused CLI.

  • 🧱 No more pluggy; new event-driven runtime
    v0.9.x replaces the old pluggy-style in-process plugin system with an event-driven, append-only-log flow. This gives us cleaner resumability, auditability, and a path toward future browser extension capture, distributed workers, and p2p sync.

  • 🗃️ Safer data layout and migrations
    The new layout keeps user-owned snapshot/crawl data under data/archive/users/{username}/..., creates crawl records for migrated snapshots, and preserves legacy ArchiveResult data instead of rediscovering everything from slow filesystem scans.

  • 🖥️ Improved browser isolation and output serving
    Chrome-based extractors now use crawl-scoped browser sessions, cleanup cloned profiles after each crawl, and serve snapshot outputs through web.archivebox.io / snap-*.archivebox.io style isolation for safer replay.

  • 🧭 Tons of UI, CLI, REST API, and database improvements
    Faster snapshot/admin list views, better live progress visibility, improved ArchiveResult detail pages, cleaner REST API paths, and more durable Process/Crawl/Snapshot database state.

What's Changed

  • 🔌 Extractor plugins moved into abx-plugins, with generated plugin docs at archivebox.github.io/abx-plugins.
  • ⚡ Archiving execution now goes through abx-dl, which can also be used directly:
    uvx abx-dl --plugins=title,screenshot,singlefile 'https://example.com'
  • 🧾 Runtime execution now writes structured append-only JSONL records for Crawls, Processes, Snapshots, and ArchiveResults.
  • 🔄 Migrations now preserve legacy Snapshot, Tag, ArchiveResult, and filesystem output data across 0.7.x / 0.8.x0.9.x upgrades.
  • 📁 Heavy crawl/snapshot data stays under data/archive/users/... for Docker volume compatibility.
  • 🌐 Added subdomain-aware replay for public web, admin, API, and isolated snapshot hosts.
  • 🔒 Added stricter public/admin/API separation and safer default demo deployment options, including disabling public add flows by default.
  • 🧩 Browser extractors now share crawl-scoped Chrome sessions and centralize profile cleanup / lock handling through chrome_utils.js.
  • 📸 SingleFile, screenshot, DOM, readability, media, git, gallery, forum, and other extractors now run as plugin hooks with explicit outputs.
  • 🛠️ Improved archivebox add, archivebox run, archivebox update, archivebox version, archivebox status, and dependency install flows.
  • 🚀 Docker image now includes the new unified runtime stack, with Sonic search managed inside the main container.
  • 🧪 Validated against large migrated collections, including the public demo dataset, with resumable migration behavior and preserved DB/file outputs.

Helpful related projects and resources:

Full Changelog: v0.8.5-rc...v0.9.31-rc

v0.7.4: Updates for Docker container's SingleFile, YT-DLP, Chrome, Node, gosu, and other dependencies

Choose a tag to compare

@pirate pirate released this 18 May 09:01
3830544

This is an update for the archivebox/archivebox:latest Docker container's internal dependencies and the 0.7.x package metadata/lockfiles.

  • chromium v147 (via playwright==1.59.0)
  • Chrome for Testing version validation fixed for Playwright Chromium builds
  • Node.js -> v24.15.0
  • gosu -> v1.19
  • yt-dlp -> 2026.03.17
  • single-file -> v2.0.83
  • curl, wget, git, ripgrep, ffmpeg
  • Python dependencies bumped (except for Django, which is still limited to >=3.1.3,<3.2)

Run archivebox version after updating and make sure any manually-installed system dependencies are up to date. Always back up your archive before upgrading.


Tip

All new development work is happening over in the v0.9.x dev branch.


Full Changelog: v0.7.3...v0.7.4

v0.7.3: Updates for Docker container's SingleFile, YT-DLP, Chrome, and other dependencies only

Choose a tag to compare

@pirate pirate released this 15 Dec 10:18
d1136d7

This is just an update for the archivebox/archivebox:latest Docker container's internal dependencies.

  • python, node
  • chrome -> v131 ⭐️
  • curl
  • wget
  • yt-dlp -> 2024.12.x ⭐️
  • single-file -> v1.1.54 ⭐️ (this update should help fix many archiving issues reported in v0.7.2)
  • readability
  • ripgrep
  • sonic -> archivebox/sonic:latest
  • git

There is no change to any of the Python code so no need to pip update to this version if you are not using Docker.
Run archivebox version if not using Docker and make sure any of your manually-installed dependencies are up-to-date .
Always make sure to back up your archive.


Tip

👾 All new development work is happening over in the v0.8.x branch ➡️


Full Changelog: v0.7.2...v0.7.3

v0.8.5-rc: Prettier + faster CLI for InstalledBinaries, Machines/NetworkInterfaces health now audit logged

Choose a tag to compare

@pirate pirate released this 03 Oct 11:37
ed2d76a

Warning

This is a BETA pre-release that improves upon the previous v0.8.4-rc ALPHA pre-release. The next stable release will be v0.9.0. The v0.8.x-rc series of releases are for collecting feedback while we make big architectural improvements to support a new public plugin marketplace + ecosystem (powered by pluggy + huey + pydantic). We want brave early adopters to help us test it! (if that's not you, wait for v0.9!)

Also I just got a new job recently and I'm about to have a kid so no promises on stable release schedule anytime soon, sorry!

⬇️ BETA Instructions: 1. backup your collection 2. install the :dev branch with docker/pip (expand for details)
  1. 🗜️ Always make a full backup before installing new BETA releases!
    Remember, this is an unstable sneak-preview in the middle of a rewrite, so it MAY DAMAGE DATA.
gzip -k ./data/index.sqlite3         # do this at least 🙏
zip -r data.bak.zip data             # OR even better: backup the entire data dir
  1. 📦 Then get the latest nightly build from Docker Hub or Pip:
docker pull archivebox/archivebox:0.8.5rc51
# OR
pip install 'git+https://github.com/ArchiveBox/ArchiveBox@v0.8.5rc51'
  1. ↗️ Then run archivebox init to upgrade your collection:
    This take several hours to migrate existing data from v0.7.x on a slower HDDs (up to ~1min/1000 URLs).
archivebox install         # make sure all package and runtime dependencies are installed & available
archivebox init            # run data migrations (slow, theoretically safe to Ctrl+C and resume, but try not to)
archivebox version         # check that everything updated properly and dependencies are installed
archivebox status          # see a health report on the collection index & snapshot directories
  1. 💬 Let us know if you find bugs or have suggestions by opening a new issue! In particular we want to hear:
  • was the upgrade/migration process smooth?
  • can you find any areas of the UI/CLI that are slow?
  • how do you like the new plugin system? (see archivebox/plugins_extractor/*) Would you contribute a new plugin?

Highlights

Screenshot 2024-10-03 at 4 34 08 AM ArchiveBox shellScreenshot 2024-10-03 at 4 33 52 AM ArchiveBox help

What's Changed

  • 📦 Deprated apt and brew install methods in favor of pip + new archivebox install cmd
  • 🌈 Much improved archivebox help, archivebox version, and archivebox shell CLI interfaces
  • ⚡️ Massive speedups to binary detection and loading at startup time
  • ✍️ New Machine, NetworkInterface, and InstalledBinary models keep an audit log of host environment changes and health
  • Many other bugfixes, speedups, and internal architecture improvements
  • Move novnc web-ui to 8081 by @agowa in #1522
  • Add OpenContainer Image Format Annotations as Labels to Docker Image by @mpgirro in #1525

New Contributors

Full Changelog: v0.8.4-rc...v0.8.5-rc

v0.8.4-rc: New background worker system w/ huey+supervisord, plugin dependency auto-installing w/ Ansible/Pyinfra

Choose a tag to compare

@pirate pirate released this 11 Sep 23:27
ee1b881

Warning

This is an ALPHA pre-release that improves upon the previous v0.8.3-rc ALPHA pre-release. The next stable release will be v0.9.0. The v0.8.x-rc series of releases are for collecting feedback while we make big architectural improvements to support a new public plugin marketplace + ecosystem (powered by pluggy + huey + pydantic). We want brave early adopters to help us test it! (if that's not you, wait for v0.9!)


Highlights

  • 🪵 moved to proper event-driven task system huey + django-huey-monitor
  • 🦸‍♂️ integrated supervisord to manage bg workers
  • 📦 integrated ansible/pyinfra (an ansible alternative) to install subdependency packages at runtime
  • ⚡️ continued switching from runserver to proper Channels + Daphne ASGI
  • 🧩 lots more plugins!
Screenshot 2024-09-12 at 3 13 58 AM examplecom archivebox add

Full Changelog: v0.8.3-rc...v0.8.4-rc

v0.8.3-rc: New UI Buttons, adding/updating is now non-blocking, Daphne ASGI, Rich CLI logs, Byte Range support, ABIDs, and more...

Choose a tag to compare

@pirate pirate released this 06 Sep 10:23
31576e2

Warning

This is an ALPHA pre-release that improves upon the previous v0.8.2-rc ALPHA pre-release. The next stable release will be v0.9.0. The v0.8.x-rc series of releases are for collecting feedback while we make big architectural improvements to support a new public plugin marketplace + ecosystem. We want brave early adopters to help us test it! (if that's not you, wait for v0.9!)


Highlights

Screenshot 2024-09-06 at 3 22 01 AM Get Title Get Missing Archive again

  • New Admin action buttons text should make it clearer what the butons do
  • Adding new URLs / clicking action buttons now runs task in a BG thread instead of running syncronously (and often timing out)
  • Added ability to click "View on site" from any object in admin to go directly to viewing the content
  • Switched archivebox server from using runserver to a proper daphne ASGI server
  • Added HTTP byte range request support (allows you to seek to the middle of a big .mp4 without downloading the whole thing)
  • Added ability to regenerate ABIDs on objects that have gone out of sync
  • New plugin system architecture is coming along, standard API for hooks now available in plugantic/base_hook.py
  • improved CLI logging output using rich for pretty colors and nicer tracebacks
  • improved HTTP request logging to filter out noisy 404/304/200 lines
  • renamed .created -> .created_at, .modified -> .modified_at, .added -> .bookmarked_at, .updated -> .downloaded_at
  • allow accessing admin change pages, API records, and archive contents by both ABID and ID (UUID)
  • add ruff linting and lots of type hint improvements with pydantic
  • improve auth and CSRF security for the new REST API (cookies no longer work for API auth, a token is appended to URLs instead)
  • bump default USER_AGENT settings to chrome v128, bump yt-dlp, singlefile, etc. versions
  • lots of other small fixes, speedups, and improvements!

Screenshot 2024-09-06 at 3 01 47 AM API Identifiers
Screenshot 2024-09-06 at 3 01 44 AM USER SQUASH


Full Changelog: v0.8.2-rc...v0.8.3-rc

v0.8.2-rc: New Snapshot UI ✨, Admin UI speedups, more REST API endpoints, Django 5.1, and bugfixes

Choose a tag to compare

@pirate pirate released this 21 Aug 03:27
9c35f3d

Warning

This was a BETA pre-release that improved upon the previous v0.8.0-rc ALPHA pre-release. This one brings us closer to a final v0.8 release and contains several core architectural improvements around how we key things with unique IDs, as well as a ✨ new Snapshot Detail UI ✨.

image

image

Changelog: v0.8.0-rc...v0.8.2-rc

v0.8.0-rc: New REST API ✨, Django 5.0, S3/B2/SMB/NFS remote storage support, VNC viewer, and more

Choose a tag to compare

@pirate pirate released this 27 Mar 00:03

WIP ALPHA pre-release for the upcoming ArchiveBox v0.8 release.

Caution

This was an ALPHA pre-release. We were promoting it a little earlier than usual because it contains ✨ lots of big new features ✨ and we want brave early adopters to help us test it!

New ArchiveBox REST APIArchiveBox Admin Webhooks UIArchiveBox Configuration Admin UIS3/B2/SMB/NFS/GDrive Remote Storage Setup

Highlights

Expand to see see more...
  • add gitea and other domains to default GIT_DOMAINS list to run git archiving on
  • check /, /data, and /data/archive in Docker and warn if running low on disk space
  • Add COOKIES_FILE support for singlefile extractor by @naoph in #1372
  • Use COOKIES_FILE to fetch page titles by @benmuth in #1364
  • Fallback to not chown'ing ./data/archive dir if it's a network mount that prevents ownership changes by @gnattu in #1312
  • Show the upgrade notification only in specific views by @benmuth in #1314
  • ability to populate is_staff and is_superuser flags at LDAP authentication by @vladimirdulov in #1335
  • Make it a little easier to run specific tests by @jimwins in #1371
  • disable chrome automatic self-updating when running headless
  • Add ability to populate is_staff and is_superuser flags during LDAP first auth
  • allow more restrictive NFS permission coercion on ./data/archive
  • bump yt-dlp, singlefile, wget, curl, and chrome versions
  • fix RESOLUTION being ignored when using Chrome headless in Docker
  • fix sorting by Size / Files in the Admin Snapshots list page UI
  • fix spinner icon showing on some Snapshots instead of favicon when only a few extractors are enabled
  • fix yt-dlp sometimes failing to archive media due to filenames being too long or containing special characters
  • fix wget extractor not finding output when :80 or :443 port is present in the original URL
  • fix /var/spool/cron/crontabs permissions when mounting it via Docker
  • fix /browsers chown on Docker armv7 entrypoint failing

COMING SOON: new sci-dl scientific paper downloader being worked on by @benmuth

New Contributors

Full Changelog: v0.7.2...v0.8.0-rc

v0.7.2: Make scheduled imports taggable, fix admin buttons, readability, Docker permissions

Choose a tag to compare

@pirate pirate released this 04 Jan 19:25
315c9f3
Web version screenshot

Get this release via pip, docker, brew, or dpkg (apt & brew releases are delayed).

# Get it with Pip on any OS (`amd64`, `arm64`, `arm/v7`)
pip install --upgrade 'archivebox==0.7.2'`
# Get it with Docker on any OS (`amd64`, `arm64`, `arm/v7`)
docker pull archivebox/archivebox:0.7.2
# Get it with brew on macOS (`amd64`, `arm64`)
brew tap archivebox/archivebox
brew install archivebox
pip install --upgrade 'archivebox==0.7.2'`
# Get it with apt on Ubuntu/Debian based systems (`any`)
wget 'https://github.com/ArchiveBox/debian-archivebox/raw/main/archivebox-0.7.1.deb'
apt install ./archivebox-0.7.1.deb
# OR
dpkg -i ./archivebox-0.7.1.deb

# then run pip install after
pip install --upgrade 'archivebox==0.7.2'`

Note: this is not packaged using "proper" debian techniques like 0.6.2 was, instead it's just a wrapper for executing pip install archivebox w/ a few extras. This is because ArchiveBox relies on some binary and dynamic dependencies (node, chrome, playwright, ffmpeg, yt-dlp, etc.) which aren't allowed in Debian packages.

(Launchpad apt ppa & brew updates coming eventually, packaging all the vendored binaries that archivebox depends on has gotten harder lately)


CLI version screenshot
# Then run this to upgrade an existing collection data dir to 0.7.2
cd ~/path/to/data/dir
archivebox init

What's Changed

  • add --tag=tag1,tag2,tag3 support to archivebox schedule command
  • allow PGID=0 root-group ownership of data dir (but PUID=0 is still not allowed)
  • improve error messages, hints, and logging about permissions issues in Docker
  • notify users when new ArchiveBox version is available on Github (thanks @benmuth!)
  • bump dependency versions (yt-dlp, chrome, readability, node, python)
  • warn when Docker / or /data volume mounts don't have any space available
  • limit to compatible python version to >= 3.8 and <= 3.11

Bug Fixes

  • fix action buttons in Snapshot admin page not showing up correctly
  • tag links immediately in first stage of archivebox add instead of at the end (so that imports that are paused or interrupted still get tagged correctly)
  • fix config variables in CHROME_USER_AGENT format string not getting interpolated properly
  • switch readability to prefer Chrome DOM dumps for article text instead of singlefile (because singlefile output is often huge and crashes readability/times out)
  • make Docker image smaller by removing unneeded docs files
  • better current version detection and remove annoying +editable string and also add BUILD_TIME
  • fix /browsers/* does not exist warning on startup

v0.7.1: Minor new features, bugfixes, and new dependency versions

Choose a tag to compare

@pirate pirate released this 04 May 05:53

Get this release via pip, docker, brew, or dpkg (apt ppa update delayed).

# Get it with Pip on any OS (`amd64`, `arm64`, `arm/v7`)
pip install --upgrade 'archivebox==0.7.1'`
# Get it with Docker on any OS (`amd64`, `arm64`, `arm/v7`)
docker pull archivebox/archivebox:0.7.1
# Get it with brew on macOS (`amd64`, `arm64`)
brew tap archivebox/archivebox
brew install archivebox
# Get it with apt on Ubuntu/Debian based systems (`any`)
wget 'https://github.com/ArchiveBox/debian-archivebox/raw/main/archivebox-0.7.1.deb'
apt install ./archivebox-0.7.1.deb
# OR
dpkg -i ./archivebox-0.7.1.deb

Note: this is not packaged using "proper" debian techniques like 0.6.2 was, instead it's just a wrapper for executing pip install archivebox w/ a few extras. This is because ArchiveBox relies on some binary and dynamic dependencies (node, chrome, playwright, ffmpeg, yt-dlp, etc.) which aren't allowed in Debian packages.

(Launchpad apt ppa update coming eventually, packaging for apt has gotten harder lately)


# Then run this to upgrade an existing collection data dir to 0.7.1
cd ~/path/to/data/dir
archivebox init

What's Changed

Lots of bugfixes, speedups, and small convenience features.

New Contributors

Expand to see the list...

Full Changelog: v0.6.2...v0.7.1