NGS Fundamentals and Quality Control

Next-Generation Sequencing (NGS) is a DNA or RNA sequencing technology that can read millions of DNA fragments simultaneously. It helps researchers study genetic variations, identify microorganisms, investigate diseases, and understand gene expression. Traditional sequencing methods generally process fewer DNA fragments at a time. NGS makes it possible to analyze large amounts of genetic information more efficiently.

However, sequencing data may contain adapter sequences, low-quality bases, and other unwanted information. These problems can affect the accuracy of downstream analysis. Therefore, researchers perform Quality Control (QC) to evaluate sequencing data and identify potential issues before further analysis.

This article explains the fundamentals of NGS, major sequencing platforms, quality control tools, read trimming, alignment, and the complete NGS data processing workflow. It is part of our bioinformatics tutorial series.

What Is Next-Generation Sequencing (NGS)?

Next-Generation Sequencing (NGS) is a high-throughput sequencing technology that determines the order of nucleotides in DNA or RNA by processing millions of fragments in parallel.

DNA contains four main nucleotide bases: adenine (A), thymine (T), guanine (G), and cytosine (C). RNA uses uracil (U) instead of thymine. (Need a refresher? See Biology Basics for Coders.)

NGS breaks genetic material into smaller fragments, sequences those fragments, and uses computational tools to analyze the resulting data.

Applications of NGS

NGS is used in several areas of biological research and healthcare:

• Disease research: Identifies genetic variations associated with diseases.
• Cancer research: Helps researchers study mutations and gene expression in cancer cells.
• Infectious disease analysis: Supports the identification and monitoring of viruses and bacteria. Related sequences can later be compared using Phylogenetic Analysis in Bioinformatics.
• Genome assembly: Helps reconstruct the genome of an organism from sequencing reads. Assembled genomes are then studied with Gene Prediction and Annotation.
• Gene expression analysis: Measures RNA-derived sequencing data to study which genes are active. See Galaxy RNA-seq Analysis and R and Bioconductor for Bioinformatics.
• Microbial research: Helps identify microorganisms present in environmental, clinical, and biological samples.

1. NGS Basics and Important Terms

Before working with NGS tools, it is important to understand the common terms used in sequencing data analysis.

1.1 What Is a Sequencing Read?

A sequencing read is a sequence of nucleotide bases obtained from a DNA or RNA fragment during sequencing. For example, the sequence ATGCGTAC represents a read containing eight nucleotide bases. A sequencing experiment can generate millions or billions of reads, depending on the platform and sequencing run.

1.2 What Is a Reference Genome?

A reference genome is a representative genome sequence used as a standard for comparing sequencing reads. Reference genomes are stored in public resources such as those described in Biological Databases, and you can learn how to download them in Accessing Public Bioinformatics Data. Researchers align reads to the reference genome to determine where they match and identify differences between the sample and the reference.

For example, a researcher studying bacterial DNA may align the sample's reads against a reference genome of the same bacterial species.

1.3 What Is Sequencing Coverage?

Sequencing coverage, also called sequencing depth, describes how many sequencing reads cover a particular position in the genome.

For example, 30× coverage means that a genomic position has been read approximately 30 times on average.

Higher coverage can improve the detection of some genetic variations, although the required coverage depends on the experiment and analysis method. For the statistics behind such decisions, see Biostatistics for Bioinformatics.

1.4 What Are Single-End and Paired-End Reads?

NGS experiments can produce single-end or paired-end reads.

• Single-end sequencing: Reads one end of a DNA fragment and produces one read per fragment.
• Paired-end sequencing: Reads both ends of a DNA fragment and produces two related reads, commonly stored in R1 and R2 files.

Paired-end sequencing provides information about both ends of a fragment, which can help improve alignment and resolve some difficult genomic regions.

1.5 What Is a FASTQ File?

A FASTQ file is a text-based file format that stores sequencing reads and their corresponding base quality scores. It is one of the formats explained in Bioinformatics File Formats.

Each FASTQ record contains four lines:

@read1 GATTACAGATTACAGATTAC + IIIIIIIIIIIIIIIIIIII

The four lines represent:

1. Read identifier: Begins with @ and identifies the read.
2. Sequence: Contains the nucleotide bases.
3. Separator: Begins with + and separates the sequence from the quality information.
4. Quality string: Contains one character for each base, representing its sequencing quality.

FASTQ files are commonly stored in compressed form, such as sample.fastq.gz, to reduce storage requirements.

To read and process FASTQ files with code, see Python for Bioinformatics and Python, Biopython and Data Libraries in Bioinformatics.

1.6 What Are Phred Quality Scores?

A Phred quality score estimates the probability that a nucleotide base has been called incorrectly during sequencing.

A higher Phred score indicates a lower probability of an incorrect base call.

The relationship is:

[ Q=-10\log_{10}(P) ]

Here, (Q) represents the Phred quality score, and (P) represents the probability of an incorrect base call.

In a standard Phred+33 FASTQ file, the character I represents Q40, while # represents Q2.

Quality scores help researchers identify unreliable bases and determine whether read trimming or additional investigation is necessary. They are shown in the FastQC report and used by trimming tools.

2. NGS Sequencing Platforms

Different sequencing platforms use different technologies to read genetic material. The main differences include read length, accuracy, throughput, cost, and the type of biological question being studied.

2.1 Illumina Sequencing

Illumina sequencing is a widely used NGS technology based on sequencing by synthesis. It identifies DNA bases by detecting signals generated as new DNA strands are synthesized.

How Does Illumina Sequencing Work?

1. DNA is fragmented into smaller pieces, and sequencing adapters are attached.
2. The fragments attach to a flow cell and form clusters through amplification.
3. Fluorescently labelled nucleotides are incorporated during DNA synthesis.
4. The sequencing instrument detects the signals and determines the base sequence.
5. The resulting reads are processed and analyzed using bioinformatics tools.

Key Features of Illumina

• Produces short reads, commonly ranging from approximately 50 to 300 bases, depending on the instrument and configuration.
• Provides high base-level accuracy under suitable sequencing conditions.
• Generates large amounts of sequencing data.
• Is commonly used for whole-genome sequencing, exome sequencing, RNA-seq, and ChIP-seq.

Example: A researcher studying an inherited disorder may sequence the coding regions of a patient's genome. Illumina reads can help identify genetic variations that may be associated with the disorder.

2.2 PacBio Sequencing

PacBio sequencing is a long-read sequencing technology that uses Single Molecule, Real-Time (SMRT) sequencing to observe DNA synthesis.

PacBio HiFi sequencing generates long reads with high consensus accuracy by reading circular DNA templates multiple times and combining the observations.

How Does PacBio Sequencing Work?

1. DNA is prepared for sequencing and converted into suitable circular templates.
2. DNA polymerase synthesizes a complementary strand in a specialized sequencing instrument.
3. The instrument detects signals produced as nucleotides are incorporated.
4. For HiFi sequencing, repeated observations of the same template are combined to generate a highly accurate consensus read.

Key Features of PacBio

• HiFi reads are commonly several kilobases to tens of kilobases long, depending on the sample and sequencing setup.
• Provides high consensus accuracy.
• Helps resolve repetitive genomic regions and structural variations.
• Is useful for genome assembly, variant detection, and transcript analysis.

Example: A researcher assembling a bacterial genome may use PacBio HiFi reads to span repeated DNA regions that are difficult to reconstruct using short reads alone.

2.3 Oxford Nanopore Sequencing

Oxford Nanopore sequencing is a sequencing technology that identifies DNA or RNA sequences by measuring changes in electrical current as nucleic acid molecules pass through nanopores.

Unlike Illumina and PacBio sequencing, nanopore sequencing does not depend on fluorescence-based detection.

How Does Oxford Nanopore Sequencing Work?

1. DNA or RNA is prepared for sequencing.
2. A molecule passes through a tiny protein pore embedded in a membrane.
3. The molecule changes the electrical current passing through the pore.
4. A basecalling program interprets the electrical signals and predicts the nucleotide sequence.

Key Features of Oxford Nanopore

• Supports a wide range of read lengths, including ultra-long reads.
• Provides sequencing data in real time.
• Offers portable devices such as MinION.
• Supports DNA sequencing and, with appropriate methods, direct RNA sequencing.
• Has read accuracy that depends on the chemistry, basecaller, and analysis method.

Example: During an infectious disease investigation, researchers can use a portable nanopore sequencer to obtain genomic information and help monitor the spread of a pathogen.

2.4 Comparison of NGS Sequencing Platforms

The choice of platform depends on the research objective, required accuracy, read length, available budget, and sample type.

3. Quality Control with FastQC and MultiQC

NGS quality control is the process of evaluating sequencing data to identify potential problems before downstream analysis.

Raw reads may contain low-quality bases, adapter sequences, unusual nucleotide distributions, or contamination. Quality control helps researchers understand these issues and decide whether additional processing is required. QC works on FASTQ files and relies on Phred quality scores.

3.1 What Is FastQC?

FastQC is a bioinformatics tool that analyzes FASTQ files and generates an HTML report describing the quality characteristics of sequencing reads. FastQC evaluates the data but does not trim or modify the original reads.

How to Run FastQC

mkdir -p qc_raw fastqc sample_R1.fastq.gz sample_R2.fastq.gz \  -o qc_raw -t 4

In this command:

• sample_R1.fastq.gz and sample_R2.fastq.gz are the input FASTQ files.
• -o qc_raw specifies the output directory.
• -t 4 uses four threads.

FastQC generates an HTML report and a ZIP file for each input file. (If these shell commands are new to you, see Linux Commands for Bioinformatics.)

Important FastQC Report Sections

Per Base Sequence Quality: Shows the distribution of quality scores across read positions. A decrease in quality toward the ends may indicate that trimming should be considered.

Per Sequence GC Content: Shows the GC-content distribution across reads. Unexpected patterns may indicate contamination or an unusual sample composition.

Adapter Content: Identifies adapter sequences that remain in the reads.

Sequence Duplication Levels: Reports repeated sequences. High duplication may be expected in some experiments, so it must be interpreted according to the data type.

Overrepresented Sequences: Lists sequences occurring more frequently than expected. These may be adapters, contaminants, or biologically abundant sequences. A suspicious sequence can be identified with BLAST.

Per Base N Content: Shows positions where the sequencer could not confidently identify a nucleotide.

Understanding FastQC Results

FastQC assigns pass, warning, or fail indicators to individual report modules. These indicators help identify potential issues but do not automatically determine whether a sample is suitable for analysis.

For example, some RNA-seq datasets show unusual sequence composition near the beginning of reads because of library preparation methods. Such warnings should be interpreted in the context of the experiment (see Galaxy RNA-seq Analysis).

Example: If a FastQC report shows declining quality near read ends and increasing adapter content, the researcher may use a trimming tool and then run FastQC again to evaluate the cleaned reads.

3.2 What Is MultiQC?

MultiQC is a reporting tool that collects results from multiple bioinformatics tools and combines them into a single report. While FastQC evaluates individual sequencing files, MultiQC makes it easier to compare quality metrics across multiple samples. MultiQC supports reports from several tools, including FastQC, fastp, samtools, and many other bioinformatics programs.

How to Run MultiQC

multiqc qc_raw -o multiqc_report

Here:

• qc_raw is the directory containing the FastQC results.
• -o multiqc_report specifies the output directory.

The main report is saved as multiqc_report.html.

Example: A researcher processes 12 sequencing samples. MultiQC makes it easier to identify a sample with unusually low read counts or poor quality compared with the other samples.

4. Read Trimming and Filtering

Read trimming removes unwanted bases or adapter sequences from sequencing reads. Read filtering removes reads that do not meet specified quality or length requirements.

These steps help prepare sequencing data for downstream analysis. However, trimming should be based on quality reports (see FastQC) and experimental requirements because excessive trimming can remove useful information.

4.1 What Is Trimmomatic?

Trimmomatic is a bioinformatics tool used to trim adapters and low-quality bases from Illumina sequencing reads (see Illumina sequencing).

It supports both single-end and paired-end data and allows users to configure multiple processing steps. For the difference between the two, see single-end and paired-end reads.

Trimmomatic Command for Paired-End Reads

java -jar trimmomatic-0.39.jar PE \  -threads 4 -phred33 \  sample_R1.fastq.gz sample_R2.fastq.gz \  R1_paired.fq.gz R1_unpaired.fq.gz \  R2_paired.fq.gz R2_unpaired.fq.gz \  ILLUMINACLIP:TruSeq3-PE.fa:2:30:10 \  LEADING:3 TRAILING:3 \  SLIDINGWINDOW:4:20 MINLEN:36

This example assumes that the JAR file and the adapter file are available at the specified paths. Their paths should be adjusted to match the installation.

4.2 What Is fastp?

fastp is a fast sequencing data preprocessing tool that performs adapter trimming, quality filtering, length filtering, and quality reporting. It can automatically detect adapters in many paired-end datasets, reducing the need to specify an adapter file manually.

fastp Command for Paired-End Reads

fastp \
 -i sample_R1.fastq.gz \
 -I sample_R2.fastq.gz \
 -o clean_R1.fastq.gz \
 -O clean_R2.fastq.gz \
 --detect_adapter_for_pe \
 --qualified_quality_phred 20 \
 --length_required 36 \
 --thread 4 \
 --html fastp_report.html \
 --json fastp_report.json

4.3 Trimmomatic vs. fastp

Both tools are useful. The best choice depends on the workflow, required trimming parameters, and reproducibility requirements.

After trimming, run FastQC on the cleaned reads and compare the results with the raw-read reports. The cleaned reads are then ready for alignment.

5. Read Alignment with BWA, Bowtie2, and HISAT2

Read alignment, also called read mapping, is the process of finding where sequencing reads match a reference genome or transcriptome. The same idea is explained at a smaller scale in What Is Pairwise Sequence Alignment?, while BLAST and Homology Search shows how to search a single sequence against a database.

Alignment helps researchers determine the genomic origin of reads and supports downstream tasks such as variant calling, gene expression analysis, and identifying genomic regions of interest.

5.1 What Is BWA?

BWA (Burrows-Wheeler Aligner) is a read alignment tool commonly used to align short DNA sequencing reads to a reference genome.

BWA-MEM is a widely used BWA algorithm for DNA reads, including whole-genome and exome sequencing data.

BWA Command Example

First, build the reference index:

bwa index reference.fa

Next, align paired-end reads:

bwa mem -t 4 reference.fa \  clean_R1.fastq.gz clean_R2.fastq.gz \  > sample.sam

In these commands:

• reference.fa is the reference genome in FASTA format (see reference genome).
• bwa index builds the reference index.
• bwa mem aligns reads to the reference.
• -t 4 uses four threads.
• sample.sam stores the alignment results (see SAM files).

Example: A researcher studying a bacterial strain can align its DNA reads against a reference genome and use the resulting alignments for downstream variant analysis.

5.2 What Is Bowtie2?

Bowtie2 is a fast and memory-efficient aligner for short DNA sequencing reads. It supports both end-to-end and local alignment.

In end-to-end alignment, the entire read must align according to the selected scoring and mismatch settings. In local alignment, a matching portion of the read can be aligned while poorly matching ends may be excluded.

Bowtie2 is commonly used in ChIP-seq, ATAC-seq, and other short-read DNA analysis workflows.

Bowtie2 Command Example

Build the index:

bowtie2-build reference.fa ref_index

This command creates an index from the reference FASTA file (reference.fa), which Bowtie2 uses during sequence alignment.

2. Align paired-end reads

bowtie2 -x ref_index \
 -1 clean_R1.fastq.gz \
 -2 clean_R2.fastq.gz \
 -S sample.sam \
 -p 4

The command uses ref_index as the reference index and aligns the cleaned paired-end reads from clean_R1.fastq.gz and clean_R2.fastq.gz. The alignment results are saved in sample.sam.

Example: In a ChIP-seq experiment, researchers align sequencing reads to the reference genome before identifying regions enriched for protein-DNA interactions.

5.3 What Is HISAT2?

HISAT2 is a splice-aware aligner commonly used for RNA sequencing (RNA-seq).

In eukaryotic organisms, RNA transcripts are often formed by joining exons after introns have been removed. Consequently, an RNA-seq read may span an exon junction, with the corresponding genomic positions separated by an intron.

HISAT2 can align reads across these splice junctions, making it suitable for many RNA-seq workflows. To run a full RNA-seq pipeline without the command line, see Galaxy RNA-seq Analysis.

HISAT2 Command Example

Build the reference index:

hisat2-build genome.fa genome_index

This command builds an index from the reference genome file (genome.fa), which HISAT2 uses to align RNA sequencing reads.

2. Align paired-end RNA-seq reads

hisat2 -p 4 --dta \
 -x genome_index \
 -1 clean_R1.fastq.gz \
 -2 clean_R2.fastq.gz \
 -S rnaseq.sam

This command aligns cleaned paired-end RNA-seq reads against the indexed reference genome and saves the alignment results in rnaseq.sam.

Example: A researcher compares RNA-seq data from healthy and diseased tissue. HISAT2 aligns reads to the reference genome, and downstream tools use the alignments to study differences in gene expression.

5.4 Comparison of BWA, Bowtie2, and HISAT2

For conventional short-read DNA alignment, BWA or Bowtie2 may be suitable depending on the experiment. For RNA-seq reads that span splice junctions, a splice-aware aligner such as HISAT2 is generally appropriate.

6. SAM, BAM, CRAM, and samtools

After alignment, the output must often be converted, sorted, indexed, and checked before downstream analysis. samtools is a command-line toolkit for working with SAM, BAM, and CRAM alignment files. It supports alignment conversion, sorting, indexing, filtering, and statistics. For more on alignment and other file types, see Bioinformatics File Formats.

6.1 What Is a SAM File?

A SAM (Sequence Alignment/Map) file is a text-based format that stores information about sequencing reads and their alignments. It includes details such as read names, mapping positions, alignment flags, and mapping quality. SAM files are readable but can become very large when a project contains millions of reads.

6.2 What Is a BAM File?

A BAM (Binary Alignment/Map) file is the compressed binary representation of SAM data. BAM files are generally smaller and more efficient to process than their text-based SAM equivalents. Sorted BAM files are commonly used by downstream bioinformatics tools.

6.3 What Is a CRAM File?

A CRAM file is a compressed format for storing sequencing alignments. It can reduce storage requirements and typically relies on the appropriate reference sequence for decoding reference-dependent information. It is useful when storage efficiency is important and the reference genome can be preserved and identified reliably.

6.4 Essential samtools Commands

Convert SAM to BAM

samtools view -b -o sample.bam sample.sam

The -b option requests BAM output, and -o specifies the output file.

Sort a BAM File

samtools sort -o sample.sorted.bam sample.bam

This command sorts the alignments by reference position.

Index a Sorted BAM File

samtools index sample.sorted.bam

This creates an index file that enables efficient access to specific genomic regions.

Check Alignment Statistics

samtools flagstat sample.sorted.bam

This command reports alignment statistics, including mapped reads and pairing information.

Filter Alignments

samtools view -b -F 4 -q 30 \  sample.sorted.bam > sample.filtered.bam

Here:

• -b requests BAM output.
• -F 4 excludes reads marked as unmapped.
• -q 30 keeps alignments with mapping quality of at least 30.

Mapping quality estimates confidence in an alignment's placement. It is not the same as the base quality stored in a FASTQ file (see Phred scores).

View a Specific Genomic Region

samtools view sample.sorted.bam NC_000913.3:1000-2000

This command displays alignments overlapping the specified region. The BAM file must be sorted and indexed for efficient regional access.

The reference name must match the sequence names in the alignment file.

Other useful commands include samtools stats, samtools idxstats, and samtools depth for additional alignment and coverage information.

7. Complete NGS Workflow

A typical short-read NGS workflow includes raw-data quality assessment, read cleaning, alignment, and alignment processing.

The following example uses paired-end DNA reads, fastp for preprocessing, BWA for alignment, and samtools for BAM processing.

The commands assume that the required tools are installed and that raw/ contains the two input FASTQ files. To follow along comfortably in the terminal, review Linux Commands for Bioinformatics.

Step 1: Create Output Directories

mkdir -p qc_raw qc_trimmed trimmed aligned

These directories separate the raw quality reports, cleaned-read reports, trimmed reads, and alignment files.

Step 2: Check Raw Reads

fastqc -t 4 -o qc_raw \  raw/sample_R1.fastq.gz \  raw/sample_R2.fastq.gz

Review the reports to identify potential quality problems and adapter contamination (see FastQC).

Step 3: Clean the Reads

fastp \
 -i raw/sample_R1.fastq.gz \
 -I raw/sample_R2.fastq.gz \
 -o trimmed/sample_R1.fastq.gz \
 -O trimmed/sample_R2.fastq.gz \
 --detect_adapter_for_pe \
 --qualified_quality_phred 20 \
 --length_required 36 \
 --thread 4 \
 --html trimmed/sample_fastp.html \
 --json trimmed/sample_fastp.json

This command produces cleaned paired-end reads and quality reports.

Step 4: Check the Cleaned Reads

fastqc -t 4 -o qc_trimmed \  trimmed/sample_R1.fastq.gz \  trimmed/sample_R2.fastq.gz

Compare the new reports with the raw-read reports to determine whether quality improved and whether any issues remain.

Step 5: Build the Reference Index

bwa index reference.fa

This step creates the index required by BWA. It generally needs to be performed only once for each reference sequence and index configuration.

Step 6: Align Reads and Sort the Output

bwa mem -t 4 \
 -R '@RG\tID:s1\tSM:s1\tPL:ILLUMINA' \
 reference.fa \
 trimmed/sample_R1.fastq.gz \
 trimmed/sample_R2.fastq.gz \
 | samtools sort -@ 4 \
     -o aligned/sample.sorted.bam -

This command aligns paired-end sequencing reads to the reference genome using BWA-MEM and sorts the alignment results using SAMtools. The final output is saved as aligned/sample.sorted.bam in BAM format.

BWA-MEM sends its SAM output directly to SAMtools through a pipe (|), so there is no need to save an intermediate SAM file.

Step 7: Index the BAM File and Check Alignment Statistics

samtools index aligned/sample.sorted.bam

samtools flagstat aligned/sample.sorted.bam \
 > aligned/sample.flagstat.txt

The first command creates the BAM index. The second saves a summary of alignment statistics to a text file. Review the mapping rate and other metrics in the context of the organism, reference genome, library preparation, and experiment.

Step 8: Combine the Reports with MultiQC

multiqc . -o multiqc_report

Run this command from the project directory after generating the reports. MultiQC searches for supported report files and combines the available results into a single HTML report (see MultiQC).

Review the report to compare samples and identify possible quality or alignment problems.

Conclusion

Next-Generation Sequencing (NGS) enables researchers to analyze large amounts of DNA and RNA sequencing data for genomic research, disease investigation, and gene expression studies. However, reliable results depend on the quality of the sequencing data and the appropriate selection of bioinformatics tools.

Quality control tools such as FastQC and MultiQC help identify potential issues in raw sequencing data, while fastp and Trimmomatic remove unwanted sequences and filter low-quality reads. Alignment tools such as BWA, Bowtie2, and HISAT2 map sequencing reads to a reference genome or transcriptome according to the analysis requirements. Finally, SAMtools helps convert, sort, index, and evaluate alignment files.