Skip to content

feature: add VFS safe wrapper#1797

Open
marco6 wants to merge 11 commits into
rusqlite:masterfrom
marco6:vfs-interface
Open

feature: add VFS safe wrapper#1797
marco6 wants to merge 11 commits into
rusqlite:masterfrom
marco6:vfs-interface

Conversation

@marco6

@marco6 marco6 commented Feb 6, 2026

Copy link
Copy Markdown
Contributor

This PR adds a the ability to write custom VFSs by implementing few safe traits.

It uses a builder pattern similar #1770 to allow registering VFSs. This way it supports all three versions of the sqlite3_file API (base, WAL-mode, fetch).

It has the following few limitations so far:

  • it is not (yet) possible to add custom file controls
  • some file controls have not been implemented yet. They are vfs-specific and I am not sure how useful that is (they could be implemented as "custom", depending on the next point)
  • it is not possible yet to get a handle to an existing VFS (so, implementing a "shim" is unsafe for now)
  • the testing API to override system calls is not implemented (xSetSystemCall, xGetSystemCall, xNextSystemCall)
  • the functions to load an extension (xDlOpen, xDlError, xDlSym, xDlClose) are not implemented for Windows
  • the custom PRAGMA file control might allocate a bit more than necessary when returning values

We also wrote a simple in-memory VFS to showcase the usage of this wrapper. We were able to successfully test this in an internal tool at Canonical, where this work took place.

This should address #1145.

@gwenn

gwenn commented Feb 6, 2026

Copy link
Copy Markdown
Collaborator

To be compared with

@marco6

marco6 commented Feb 6, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

WASM is not running properly. Any guidance on that would be appreciated as I never targeted any WASM target not in Rust or in any other language.

Regarding other VFSs crates, please do compare.

I think we are way ahead in:

  • supporting file controls (including PRAGMAs)
  • supporting all file versions (including WAL, which AFAICS is only supported as an experiment by sqlite-vfs and is mapping things in a significantly different way than what it is expected by normal SQLite operation; EDIT: Fetch API is not supported in any of those AFAICS)
  • care about lifetimes (for example sqlite-vfs can only register, but can't unregister. same for sqlite-plugin)

But please share your feedback when you have some time.

@marco6

marco6 commented Feb 6, 2026

Copy link
Copy Markdown
Contributor Author

WASM is now compiling. I think that VFSs tests are not running though.

@Spxg

Spxg commented Feb 9, 2026

Copy link
Copy Markdown
Contributor

WASM is now compiling. I think that VFSs tests are not running though.

I took a quick look, and your VFS uses some things that are not supported by wasm, such as Systemtime and thread::sleep. If you want to try running tests, you can change #[test] to #[wasm_bindgen_test] like other tests.

WASM now uses memvfs.

@marco6

marco6 commented Feb 18, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

WASM is now compiling. I think that VFSs tests are not running though.

I took a quick look, and your VFS uses some things that are not supported by wasm, such as Systemtime and thread::sleep. If you want to try running tests, you can change #[test] to #[wasm_bindgen_test] like other tests.

WASM now uses memvfs.

I decided for now to remove support for WASM. I do not have enough experience with it to sensibly deal with that and I think it would just make this PR even bigger.

I think support can be added as a separate PR, once we are ok (or not!) with the interface proposed here.

@marco6

marco6 commented Feb 18, 2026

Copy link
Copy Markdown
Contributor Author

I tried to implement the windows xDl* methods. I need to think about how to test this, but hopefully it won't be too difficult.

@marco6

marco6 commented Feb 18, 2026

Copy link
Copy Markdown
Contributor Author

Would it be better if I used something like libloading instead? I didn't want to add more dependencies...

@codecov

codecov Bot commented Mar 12, 2026 •
edited
Loading

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 38.11293% with 833 lines in your changes missing coverage. Please review.
✅ Project coverage is 82.41%. Comparing base (6d3c282) to head (eb15eda).
⚠️ Report is 4 commits behind head on master.

Files with missing lines Patch % Lines
src/vfs/mod.rs 38.19% 830 Missing ⚠️
src/error.rs 0.00% 3 Missing ⚠️
Additional details and impacted files 
@@            Coverage Diff             @@
##           master    #1797      +/-   ##
==========================================
- Coverage   87.69%   82.41%   -5.29%     
==========================================
  Files          58       59       +1     
  Lines       10948    12309    +1361     
==========================================
+ Hits         9601    10144     +543     
- Misses       1347     2165     +818

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@marco6 @gwenn @Spxg @TheSignPainter98