The unipept index written entirely in Rust. This repository consists of multiple different Rust projects that depend on each other. More information about each project can be found in their respective README.md file.

Clone this repository by executing the following command:

git clone https://github.com/unipept/unipept-index.git

Next, build everything by executing the following command in the root of the repository.

cargo build --release

After a successful build, two executable binaries are available under target/release/:

• sa-builder — builds the index from a protein database file
• sa-server — serves peptide search queries over HTTP

sa-builder constructs a suffix array index from a protein sequence database. It reads a tab-separated input file, builds a (sparse, optionally compressed) suffix array over the concatenated protein sequences, and writes three binary output files: the suffix array itself, the serialised protein metadata, and a suffix-to-protein mapping. These three files are the input required by sa-server.

sa-builder [OPTIONS] --database-file <DATABASE_FILE> --output-sa <OUTPUT_SA> --output-proteins <OUTPUT_PROTEINS> --output-mapping <OUTPUT_MAPPING>

The database file must be a tab-separated file (TSV) with one protein per line and four columns:

<uniprot_accession>  <taxon_id>  <amino_acid_sequence>  <functional_annotations>

Functional annotations are a semicolon-separated list of terms such as GO:0009279;IPR:IPR016364.

• lib-sais (default): Uses the libsais algorithm. Supports native sparse suffix array construction, which is more memory-efficient than building a full array and sampling it afterwards. The maximum effective sparseness that libsais handles natively is 5; for larger sparseness factors the tool applies an additional sampling step on top of libsais output.
• lib-div-suf-sort: Uses the libdivsufsort algorithm. Always builds the full (non-sparse) suffix array first, then applies the sparseness sampling step afterwards. This requires more memory during construction when a sparseness factor greater than 1 is used.

The mapping file records which text position belongs to which protein. Three representations are available:

• bit-vec (default): Stores one bit per text position using a Rank9 rank/select data structure. Separator and terminator characters are marked with a 1-bit; protein characters with a 0-bit. Protein lookup is O(1) via a rank query. Memory usage is approximately 2 bits per character. This is the recommended option.
• dense: Stores a 32-bit protein index for every text position (4 bytes per character). Lookup is O(1) by direct array access. Higher memory and file size than bit-vec, but conceptually simpler.
• sparse: Stores only the starting text position of each protein (8 bytes per protein). Lookup is O(log m) where m is the number of proteins, via binary search. Smallest file size of the three options, but slower lookup.

sa-builder \
  --database-file uniprot_proteins.tsv \
  --output-sa index.sa \
  --output-proteins index.proteins.bin \
  --output-mapping index.mapping \
  --sparseness-factor 3 \
  --construction-algorithm lib-sais \
  --compress-sa \
  --mapping-style bit-vec

sa-server loads the three binary files produced by sa-builder and exposes an HTTP API for peptide search. It accepts POST requests containing a list of peptide sequences and returns all matching proteins from the index along with their UniProt accession, taxon ID, and functional annotations.

The server binds to 0.0.0.0:3000 and is ready to accept requests once all index files are loaded.

sa-server [OPTIONS] --database-file <DATABASE_FILE> --index-file <INDEX_FILE> --mapping-file <MAPPING_FILE>

POST /search

Accepts a JSON body and returns a JSON array of search results.

Request body fields:

Response: a JSON array where each element corresponds to one peptide that had at least one match. Peptides with no matches are omitted from the response.

[
  {
    "sequence": "MSKIAALLPSV",
    "cutoff_used": false,
    "proteins": [
      {
        "taxon": 9606,
        "uniprot_accession": "P12345",
        "functional_annotations": "GO:0005737;IPR:IPR016364"
      }
    ]
  }
]

sa-server \
  --database-file index.proteins.bin \
  --index-file index.sa \
  --mapping-file index.mapping \
  --mmap

curl -X POST http://localhost:3000/search \
  -H "Content-Type: application/json" \
  -d '{
    "peptides": ["MSKIAALLPSV", "ACDEFGHIK"],
    "equate_il": true,
    "tryptic": false,
    "cutoff": 5000
  }'