BigBacter Database

1. Overview
2. Database Structure
   1. Directory Structure Explanation
3. Taxon-Level Files
   1. Floc Signatures (sig/)
      1. Purpose
      2. Contents
   2. Database Config (config.json)
      1. Purpose
      2. Contents
      3. Checks
4. Cluster-Level Files
   1. Assembly Files (asm/)
      1. Purpose
      2. Contents
   2. Auxiliary Files (aux/)
      1. Purpose
      2. Contents
   3. Variant Files (var/)
      1. Purpose
      2. Contents

Overview

BigBacter maintains a structured database of genomic files required for routine bacterial surveillance analysis. Each execution compares new samples from the input samplesheet (--input) against historical samples stored in the database (--db).

Samples are only saved to the database when you run BigBacter using the --push true parameter.

This page details the database structure, file formats, and the purpose of each component within a BigBacter database.

Database Structure

The BigBacter database follows a hierarchical organization based on taxonomic classification and clustering results:

${db}/
└── ${taxa}/
    ├── config.json
    ├── clusters/
    │   └── ${cluster}/
    │       ├── asm/
    │       │   ├── ${sample}.fa.gz
    │       │   └── ref.fa.gz
    │       ├── aux/
    │       │   └── ref.json.gz
    │       └── var/
    │           └── ${sample}.tar.gz
    └── sig/
        └── ${sample}.sig.gz

Directory Structure Explanation

• ${db}: Root database directory
• ${taxa}: Species-level taxonomic identifier (e.g., *Escherichia coli*)
• clusters/${cluster}: Cluster-specific data grouped by genomic similarity using Floc (e.g., clusters/1)
• sig/: Species-level signature files for cluster assignment
• config.json: Clustering settings the taxa database was created with (v2.2.0+)

---

Taxon-Level Files

Floc Signatures (sig/)

Purpose

Species-wide genomic signatures used for cluster assignment of new samples.

Contents

File	Description
${sample}.sig	Sourmash MinHash signatures generated by Floc in JSON format

Example

[
  {
    "class": "sourmash_signature",
    "email": "",
    "hash_function": "0.murmur64", 
    "filename": "1",
    "name": "Sample01",
    "license": "CC0",
    "signatures": [...],
    "version": 0.4
  }
]

Floc repurposes the filename field in the Sourmash signature to store cluster information.

Database Config (config.json)

New in v2.2.0.

Purpose

Records the clustering settings a taxa database was created with, so later runs cluster new samples the same way.

Contents

Field	Description
clust_dist	Distance threshold (--clust_dist) the database was built with
clust_assign_hashfreq	Assign hash frequency (--clust_assign_hashfreq) the database was built with
floc_version	Floc version used to create the database
bigbacter_version	BigBacter version used to create the database

Example

{
  "clust_dist": 0.1,
  "clust_assign_hashfreq": 0.6,
  "floc_version": "1.2.4",
  "bigbacter_version": "2.2.0"
}

config.json is written only by the --push true run that creates the taxa database. It is saved uncompressed to <db>/<taxa>/config.json, with a copy in <outdir>/<timestamp>/<taxa>/config.json. Later runs do not write or push it again.

Checks

Before clustering against an existing taxa database (one with a sig/ directory), BigBacter reads its config.json:

Condition	Result
config.json is missing	Run fails
--clust_dist differs from clust_dist	Run fails
--clust_assign_hashfreq differs from clust_assign_hashfreq	Warning only

Databases created before v2.2.0 have no config.json, so runs against them fail. We strongly recommend rebuilding them with the new defaults: point --db at a new, empty location and run all previously processed samples through BigBacter v2.2.0 with --push true. Cluster names and membership will likely change. See this post for details and a temporary workaround.

---

Cluster-Level Files

Each cluster maintains three distinct data types organized into separate directories:

Assembly Files (asm/)

Purpose

Storage of genomic assemblies in FASTA format for reference and sample comparison.

Contents

File	Description
ref.fa.gz	Cluster-specific reference assembly used for core genome analysis
${sample}.fa.gz	Individual sample assemblies (not currently used - retained for future dev)

Reference assemblies are immutable (cannot be changed)!

Auxiliary Files (aux/)

Purpose

Auxillary / supplemental information storage.

Contents

File	Description
ref.json.gz	Reference metadata, including the original sample name, number of contigs, and length

Example:

{
  "length": 4840898,
  "n_contigs": 3,
  "name": "Sample01"
}

Variant Files (var/)

Purpose

Comprehensive variant analysis data for core genome phylogenetics.

Contents

File	Description
${sample}.tar.gz	Compressed tarball (.tar.gz) containing the minimal Snippy output needed for core SNP analysis

---