Research Article |
Corresponding author: Valentin Vasselon ( valentin.vasselon@scimabio-interface.fr ) Corresponding author: Benoît Paix ( benoit.paix@inrae.fr ) Academic editor: Tiina Laamanen
© 2025 Valentin Vasselon, Sinziana F. Rivera, Éva Ács, Salomé F.P. Almeida, Karl B. Andree, Laure Apothéloz-Perret-Gentil, Bonnie Bailet, Ana Baričević, Kevin K. Beentjes, Juliane Bettig, Agnès Bouchez, Camilla Capelli, Cécile Chardon, Mónika Duleba, Tina Elersek, Clémence Genthon, Maša Jablonska, Louis Jacas, Maria Kahlert, Martyn G. Kelly, Jan-Niklas Macher, Federica Mauri, Marina Moletta-Denat, Andreia Mortágua, Jan Pawlowski, Javier Pérez-Burillo, Martin Pfannkuchen, Erik Pilgrim, Panayiota Pissaridou, Frédéric Rimet, Karmen Stanic, Kálmán Tapolczai, Susanna Theroux, Rosa Trobajo, Berry Van der Hoorn, Marlen I. Vasquez, Marie Vidal, David Wanless, Jonathan Warren, Jonas Zimmermann, Benoît Paix.
This is an open access article distributed under the terms of the CC0 Public Domain Dedication.
Citation:
Vasselon V, Rivera SF, Ács É, Almeida SFP, Andree KB, Apothéloz-Perret-Gentil L, Bailet B, Baričević A, Beentjes KK, Bettig J, Bouchez A, Capelli C, Chardon C, Duleba M, Elersek T, Genthon C, Jablonska M, Jacas L, Kahlert M, Kelly MG, Macher J-N, Mauri F, Moletta-Denat M, Mortágua A, Pawlowski J, Pérez-Burillo J, Pfannkuchen M, Pilgrim E, Pissaridou P, Rimet F, Stanic K, Tapolczai K, Theroux S, Trobajo R, Van der Hoorn B, Vasquez MI, Vidal M, Wanless D, Warren J, Zimmermann J, Paix B (2025) Proficiency testing and cross-laboratory method comparison to support standardisation of diatom DNA metabarcoding for freshwater biomonitoring. Metabarcoding and Metagenomics 9: e133264. https://doi.org/10.3897/mbmg.9.133264
|
DNA metabarcoding of benthic diatoms has been successfully applied for biomonitoring at the national scale and can now be considered technically ready for routine application. However, protocols and methods still vary between and within countries, limiting their transferability and the comparability of results. In order to overcome this, routine use of DNA metabarcoding for diatom biomonitoring requires knowledge of the sources of variability introduced by the different steps of the procedure. Here, we examine how elements of routine procedures contribute to variability between European laboratories. A set of four experiments were performed focusing on DNA extraction and PCR amplification steps to evaluate their reproducibility between different laboratories and the variability introduced by different protocols currently applied by the scientific community. Under the guidance of a reference laboratory, 17 participants from 14 countries performed DNA extraction and PCR amplification in parallel, using the same fixed protocol and their own choice of protocol. Experiments were performed by each participant on a set of standardised DNA and biofilm samples (river, lake and mock community) to investigate potential systematic and random errors. Our results revealed the successful transferability of a protocol amongst labs and a highly similar and consistent ecological assessment outcome obtained regardless of the protocols used by each participant. We propose an “all for one but prove them all” strategy, suggesting that distinct protocols can be used within the scientific community, as long as their consistency is be proven by following minimum standard requirements.
Cross-laboratory experiment, DNA-based approach, ecological status assessment, intercalibration, standardisation
Genetic approaches have recently emerged for monitoring biodiversity in aquatic ecosystems (
Significant progress has been made in developing DNA metabarcoding techniques for assessing benthic diatom assemblages. Technological advancements and optimised protocols have allowed the acquisition of robust taxonomic (species) and genetic (Operational Taxonomic Unit - OTU, Amplicon Sequence Variant - ASV) information, from which diatom quality indices can be calculated in order to infer ecological status of rivers and lakes (
Intercalibration exercises and proficiency tests have been long used to harmonise knowledge and results within the scope of the WFD. While these exercises are standard practice for morphological intercomparison (e.g.
Building on previous initiatives that led to the development of a CEN technical report for the routine sampling of benthic diatoms from rivers and lakes adapted for metabarcoding analyses (
To this end, 17 laboratories from 14 countries performed DNA extraction and PCR amplification simultaneously, under the guidance of a reference laboratory agreed by the participants (INRAE CARRTEL, France). Participants followed stringent quality control measures and adhered to established standards and protocols. Each utilised both an agreed standard lab protocol and their own choice of protocol. Each participant conducted experiments on a standardised set of DNA and biofilm samples, encompassing samples from a river, a lake and a mock community.
In order to assess the variability in DNA extraction and PCR amplification steps, all other metabarcoding process steps were standardised and MiSeq sequencing preparation was performed by the reference laboratory. Variability within and between participants was assessed in terms of DNA extract quantity, taxonomic (genus, species) and genetic richness, as well as community structure comparison and diatom quality index scores (IPS and IBD). Additionally, the impact of the different DNA extraction and PCR amplification protocols on the diatom quality index scores and the final ecological status assessment was examined.
The objective of this collaborative effort extended beyond assessing individual participant performance. We aimed to determine whether a molecular method could be readily transferred between different laboratories for potential operational use in routine freshwater monitoring. While the results do not aim to prescribe a universally applicable protocol, they provide valuable insights for establishing guidelines and minimum requirements for conducting diatom metabarcoding for biomonitoring. The primary focus of the cross-laboratory experiments is to evaluate method variability and transferability in a controlled environment, rather than favouring any specific laboratory or molecular technique.
Organisation of a collaborative international cross-laboratory experiment focusing on diatom DNA metabarcoding for biomonitoring was agreed during the COST DNAqua-Net WG2 workshop held in CUT, Limassol (Cyprus) in 2019. A core organising team, composed of CUT, OFB and Scimabio Interface, along with the reference laboratory (RL, affiliation: INRAE CARRTEL), was designated to lead and coordinate the experiments. Seventeen laboratories from 14 countries (referred to as A to Q) within the international DNAqua-Net community expressed interest in participating in the experiments. These laboratories represented both public and private institutes, with varying levels of expertise in the application of molecular methods. Additionally, the laboratories had diverse experience in applications in research and operational freshwater monitoring. Major service providers were also invited to contribute to the experiments (e.g. Macherey-Nagel, Takara).
Two categories of international cross-laboratory experiments were conducted: proficiency tests and method comparison. The “proficiency tests” experiments were designed to assess the performance of each participant using the same molecular protocols (DNA extraction and PCR amplification), thus providing information regarding the transferability and the reproducibility of the tested protocol. The “method comparison” experiments were designed to evaluate the variability introduced when using different molecular protocols (DNA extraction, PCR amplification), thus providing insights into the flexibility of molecular approaches. The organisation of cross-laboratory experiments involved several key phases (Fig.
Workflow of the experimental design of the study (experiments E1-E4). This details the steps of the proficiency test (blue), the method comparison (orange) and those related to the organisation of the whole experiment by the reference lab (RL, green). L, R, M and W labels correspond to the lake sample, river sample, mock community and negative controls, respectively.
Phase 1: The core organising team designed the experiments and circulated laboratory protocols (DNA extraction, PCR amplification) and reagents (DNA extraction and Polymerase kits, metabarcoding primers, batch control) to all participating laboratories, along with calibrated samples preparation (biofilm, DNA extracts, see details in section Collection and preparation of samples). Each participant communicated their interest in contributing to the various experiments (E1 to E4, described below) to both the core team and the reference laboratory, which subsequently provided them with the corresponding experimental materials (Fig.
Phase 2: After receiving the experimental materials, independent testing was carried out simultaneously by all participants and the reference laboratory. The experiments were conducted in parallel within a one-month period to minimise the potential effect of DNA and biofilm sample degradation.
Phase 3: All experimental outputs (data) and products (DNA extracts, PCR amplicons) were submitted by each participant to the reference laboratory, which finalised library preparation prior to high-throughput sequencing (HTS) and shipped them to the sequencing platforms.
Phase 4: After production of HTS data (fastq), bioinformatics treatments and downstream data analyses were performed by the reference laboratory.
To follow proficiency test statistical design (
We performed two experiments focused on the DNA extraction (E1) and PCR amplification (E2) steps to assess the consistency of the results obtained from different labs using the same samples and reagents. For brevity, we refer to these as proficiency tests. To simplify the experiments, the protocols used for DNA extraction from biofilm samples and PCR amplification of diatom communities corresponded to those routinely used by the reference laboratory for diatom metabarcoding and already applied by the scientific community (e.g.
DNA extraction proficiency test experiment (E1): All participants applied the Nucleospin Soil kit (Macherey-Nagel, https://dx.doi.org/10.17504/protocols.io.bd52i88e) to extract DNA from four calibrated samples: river biofilm (R), lake biofilm (L), mock community (M) and DNA-free water (W) as the negative control (Table
Experimental design of the different experiments. RL: Reference Laboratory. C - control, W – water.
Objective | Experiment | No. of protocols | No. of participants | No. of replicates | Calibrated samples | ||||
---|---|---|---|---|---|---|---|---|---|
Type | Lake (L) | River (R) | Mock (M) | C- (W) | |||||
Proficiency test | E1 - DNA extraction | 1 | 17 | 1 | Biofilm | 1 | 1 | 1 | 1 |
E2 - PCR amplification | 1 | 17 | 1 | DNA | 1 | 1 | 1 | 1 | |
Method comparison | E3 - DNA extraction | 9 + RL | 9 | 3 | Biofilm | 3 | 3 | 3 | 3 |
E4 - PCR amplification | 9 + RL | 9 | 3 | DNA | 3 | 3 | 3 | 3 | |
Reference | E1, E2, E3, E4 | 1 | 1 (RL) | 3 | Biofilm | 3 | 3 | 3 | 3 |
PCR amplification proficiency test experiment (E2): All participants applied the PCR protocol (https://dx.doi.org/10.17504/protocols.io.bd94i98w) to amplify the 312 bp rbcL barcode (
We made comparisons of methods related to DNA extraction (E3) and PCR amplification (E4) steps by asking participants to use their own choice of protocols on a set of calibrated samples. Nine of the 17 participants had an internal protocol different from the reference laboratory for E3 and E4 and, therefore, participated in these experiments. Detailed protocols were not collected and only key steps and characteristics were collected to maintain anonymity of participants (Suppl. material
DNA extraction experiments (E3): These nine participants used their own choice of DNA extraction methods and protocols on the same calibrated samples used for E1 (R, L, M, W). As a result, a significant number of parameters were expected to vary amongst participants (e.g. the lysis method, the use of proteinase K and RNase A and the purification method, Suppl. material
PCR amplification experiments (E4): Nine participants applied their own choice of PCR amplification protocol to amplify the 312 bp rbcL (263 bp barcode and the primers) on the same DNA calibrated samples used for E2 (R, L, M, W). As for E3, several parameters are expected to vary, especially according to the distinct polymerase used (e.g. exonuclease activity, hot start activity, sensitivity, fidelity, efficiency, Suppl. material
The return shipments from all participants totalled 68 DNA extracts (4 samples, 1 replicate, 17 participants) from the E1 experiments and 108 DNA extracts (4 samples, 3 replicates, 9 participants) from the E3 experiment. The reference laboratory (RL) performed rbcL 312 bp PCR amplification on each DNA extract in triplicate, following the same protocol as described previously for the E2 experiments. PCR triplicates were then pooled together to obtain one pool of amplicon per sample. Additionally, a total of 68 pools of amplicons (4 samples, 1 pool replicate, 17 participants) from E2 experiment and 108 (4 samples, 3 pool replicates, 9 participants) from E4 experiment were also received. RL produced a total of 12 DNA extracts (4 samples, 3 replicates) corresponding to 12 pools of amplicons after PCR amplification. A total of 364 individual pools of amplicons were obtained from all experiments. Their quality was controlled using gel electrophoresis; negative controls (W) were not sequenced, except for 3 W samples (E1_E, E1_Q, E3_O) when unexpected amplification was observed (potential contamination). The amplicon quality of one sample from the E2 experiment (participant G, L sample) was considered to be too low (faint amplicon band in gel electrophoresis) and not sequenced.
Altogether, a total of 275 successful PCR reactions from E1, E2, E3 and E4 experiments were sent to the INRAE Transfert sequencing facility (Toulouse, France), which carried out the final library preparation for MiSeq pair-end (PE) 2*250 bp (from the PCR2 to the equimolar pooling of the amplicons). Three samples from the E4 experiment (2 replicates of M sample from participant P and 1 replicate of R sample from participant E), produced less than 1000 reads after bioinformatics treatments. In order to conserve the statistical design, they were sequenced in a second MiSeq PE 2*250 bp sequencing run. All the samples and replicates produced by the RL were included in both runs to evaluate potential variability introduced by the two runs prior validating the use of those three samples during downstream analysis.
A total of 9,369,334 raw reads were obtained from the first MiSeq PE sequencing runs along with a further 257,686 reads from the three samples from the second MiSeq PE run. The raw data consisted of demultiplexed fastq files pairs (R1.fastq and R2.fastq) per sample accessible on the NCBI Sequences Read Archive (SRA) under the BioProject accession numbers PRJNA1187555 for experiments E1 and E3 and PRJNA1187576 for E2 and E4. Bioinformatics treatments of raw MiSeq demultiplexed data were performed using DADA2 (
Although the original objective was to perform morphological intercalibration experiments involving all participants using the same calibrated samples, major coordination challenges were encountered (see Discussion). Consequently, only the reference laboratory performed the morphological determination, to provide a first comparison of the IBD and IPS scores and the intra-operator variability obtained through this additional method following European and French standards (
Several key metrics were computed for all samples: DNA concentration [DNA] for E1, number of DNA reads and diversity metrics (of ASVs and Species) expressed with Hills numbers (q = 0, q = 1, q = 2) for both E1 and E2. Briefly, Hills numbers mathematically unify diversity concepts and are well adapted to comparison of DNA metabarcoding outputs (
The molecular and morphological inventories obtained from the river and lake samples were used to compare diatom community composition obtained by each participant in experiments E1 and E2. Additionally, the resulting molecular and morphological inventories were also used to compute the Specific Pollution Sensitivity index (IPS) (
In order to evaluate the performance and consistency of all participants (including RL) in the proficiency tests, z-scores were computed, based on the IPS scores for R and L samples. z-scores, also called “standard score”, show how far the IPS value of one participant is from the mean IPS of all participants and is calculated as follows: z = (x – μ) / σ where x is the IPS value of the participant, μ is the mean value of all participants and σ the corresponding standard deviation. Z-scores usually ranged from -3 SD to +3 SD. In the context of a proficiency test, any Z-score value < -3SD or > +3SD indicates that the protocol tested is not reproducible between the participants and has failed the proficiency test (
As IPS z-scores were computed for each participant on two environmental samples (R, L), then they can be projected one to another as a Youden plot (
To evaluate the consistency of all participants in applying their own choices of DNA extraction and PCR protocols, the diatom assemblages obtained by each participant for each replicate were analysed using histograms, with ASVs gathered at the species level. Non-metric multidimensional scaling (nMDS) was used to assess variation in diatom composition both between and within participants visually. The nMDS was performed on Bray-Curtis distances of diatoms, based on relative abundance using the vegan package (
An online workshop was organised on 21 March 2021 to present preliminary results to the participants and collect feedback on the experiment. All participants successfully completed the experiments within the expected timeframe using the materials and protocols provided. Despite the application of new DNA extraction (E1) and PCR amplification (E2) protocols on blind calibrated samples, the feedback was highly positive in terms of their accessibility and operability, even for the molecular biology laboratories which defined themselves as “less experienced”. A wide range of methods were employed by individual laboratories for DNA extraction (E3) and PCR amplification (E4) (Suppl. material
The quality and quantity of DNA extracted during experiments E1 and E3 were assessed by the participants using different methods and devices, hindering direct comparisons. Consequently, the reference laboratory quantified DNA extracts using a standardised method (Picogreen), except for one participant (E), who did not have any DNA available for quantification. Variability was observed in DNA concentration obtained by the 17 participants and the reference laboratory (n = 18) for the river, lake and mock samples with an average of 43.7 ng µl-1 (SD: ± 27.8), 25.2 ng µl-1 (SD: ± 12) and 1.2 ng µl-1 (SD: ± 1.6) respectively.
All PCR amplifications conducted during the E2 and E4 experiments were successfully completed by the participants, meeting the minimum quality requirements for rbcL amplicons set by sequencing facilities for MiSeq sequencing. However, during library preparation, unexpected amplification was observed in six of the negative controls (W samples). These samples were considered as “W false positive samples” and were consequently included in the MiSeq runs.
After bioinformatics filtering steps, a total of 275 samples representing a total of 3,029,482 DNA reads were retained, corresponding to: (i) the L, R and M biofilm samples sent to the participants for E1 (51) and E3 (81), (ii) the L, R and M DNA samples sent to the participants for E2 (50) and E4 (81), (iii) the samples prepared by the reference laboratory for comparison in all experiments (9) and (iv) the three “W false positive samples”. The composition of two “W false positive samples” were associated with E1 and corresponded exactly to a composition of an “L” sample, while the third one was associated with E3 and corresponded to an “R” sample. Consequently, these errors might be related to a pipetting error resulting to a mis-transferred sample in the W tubes, occurring before or after the DNA extractions. These errors were considered as external to the variability introduced during the protocol and, consequently, these “W false positive samples” were removed from the analysis. The resulting ASVs and corresponding molecular diatom species inventories, expressed as proportions of total DNA reads, were generated and used for downstream analyses (available on Zenodo repository system). Morphological diatom species inventories were produced by the reference laboratory (Suppl. material
The diatom assemblage composition obtained from the river (R) and lake (L) biofilm samples showed high similarity amongst the 17 participants and the reference laboratory when using the same DNA extraction (E1) and PCR amplification (E2) protocols (Fig.
Histograms of the diatom community composition (at the species level), for each of the four experiments (E1 to E4). Sample labels correspond to participant codes. RL corresponds to the reference laboratory.
However, one exception was observed for participant I (E2), where an unexpected diatom composition was obtained for the L sample, corresponding to a 1:1 mix of the L and M communities. As this non-systematic error was only detected from one participant in only one condition, it might occur during a transfer of sample, either during: (i) the sample aliquot preparation by the reference laboratory, (ii) the sample manipulation by the participant after the extraction or (iii) the HTS library preparation. Consequently, this error could not be related to the variability introduced during the application of the protocol within the participant environment (e.g. equipment, lab decontamination, manipulator practices, general contamination) as it is addressed during the proficiency test. As this “outlier” will affect the computation of z-score and the result of the proficiency test, this sample was excluded from subsequent analyses. When analysing the complete dataset, taxa with relative abundance per sample higher than 0.71% for E1 and 0.96% for E2 were successfully detected by all participants for both environmental samples (R, L). However, detection varied amongst participants for taxa below these abundance thresholds, indicating differences in their detection probability within the samples.
The concentration of DNA exhibited the highest CV amongst participants, with values of 66.2% for river samples and 48.5% for lake samples. However, for all other metrics, CV values were below 15% (Fig.
Radar charts presenting the coefficient of variations for the different metrics for the river “R” (Black line/dot) and lake “L” (Red line/dot) samples related to the Hill’s numbers (for E1 and E2), the DNA concentration (for E1) and the DNA reads numbers (for E2).
IBD and IPS scores were computed from the molecular taxonomic lists for E1 and E2 environmental samples. Consistent IBD scores of 20 (SD: 0) were obtained amongst all participants for both lake (L) and river (R) samples. The absence of variation in IBD scores between participants indicated a successful proficiency test for E1 and E2 experiments concerning this index. However, some variation was observed amongst IPS scores, with IPS deltas between participants of 0.8 for E1-lake (median: 17.1, SD: ± 0.21), 0.7 for E1-river (median: 15.9, SD: ± 0.17), 0.5 for E2-lake (median: 17.2, SD: ± 0.12) and 0.7 for E2-river (median: 15.8, SD: ± 0.16) (Fig.
IPS scores obtained for E1 and E2 experiments calculated on the river and lake samples. Z-score were computed for each participant and condition, green box indicate a z-score between [- 2SD; + 2SD], while a yellow box indicated a z-score between [- 3SD; - 2SD] and [+ 2SD; + 3SD]. The proficiency test was considered successful as no participant had a z-score] - 3SD; + 3SD]. The RL was considered here as a participant and not as a validation reference for “true values”.
Youden plots were generated for the E1 and E2 experiments using IPS z-scores obtained for river (R) and lake (L) samples (Fig.
Youden plots performed with the IPS z-scores obtained for the river and lake samples. Red square highlights the 3 standard deviation (SD) limit and the yellow box the 2 SD limit. The 45-degree reference line helps to visualise if participants have a systematic error (point close to the reference line and outside the red box), total error (point far from the reference line and outside the 3 SD box) or a random error (point far from the reference line, but within the 3 SD box).
The results obtained from the E3 (DNA extraction) and E4 (PCR method) experiments revealed a high degree of similarity in diatom assemblage composition amongst the technical replicates produced by the nine participants and the reference laboratory (Fig.
Comparison of diatom assemblage structures in E3 and E4 experiments was performed using Bray-Curtis dissimilarity indices. Between-participant variability remained low as depicted in the nMDS plot (Fig.
NMDS plots performed with the Bray-Curtis dissimilarity index calculated with the datasets obtained from both E3 and E4 experiments. Sample labels correspond to the participant codes. RL corresponds to the reference laboratory.
As participants employed their own choice of DNA extraction (E3) and PCR amplification (E4) methods, we also explored the potential impact of these procedures on diatom assemblage structure through the nMDS and using a PERMANOVA test. In the E3 experiments, the diverse cell lysis methods incorporated in DNA extraction protocols, including enzymatic (river: 66.1%, lake: 72.2%) and mechanical (river: 11.4%, lake: 20%) approaches, yielded significant PERMANOVA results with respect to the total variance observed between samples (Suppl. material
Ecological assessment was performed by computing IBD and IPS scores using molecular taxonomic lists for E3 and E4 environmental samples. As for the E1 and E2 experiments, consistent IBD scores of 20 (SD: 0) were obtained for lake and river samples across all participants and replicates. However, in the E3 experiments, IPS scores exhibited significant variation amongst participants for both river (Kruskal-Wallis, p = 0.002) and lake (Kruskal-Wallis, p = 0.001) samples, with an IPS delta between participants of 1.1 for lake (median: 17, SD: ± 0.37) and 1.3 for river (median: 15.7, SD: ± 0.38) samples. Similarly, in the E4 experiments, IPS scores also displayed significant differences between participants for river (Kruskal-Wallis, p = 0.001) and lake (Kruskal-Wallis, p = 0.001) samples, with an IPS delta between participants of 0.7 for lake (median: 17.3, SD: ± 0.21) and 1.2 for river (median: 16, SD: ± 0.24) (Fig.
Diatom taxa from both lake and river samples were also identified in triplicate by the reference laboratory using their morphological characteristics (Suppl. material
Through this study, we first compared the results obtained from 17 participating laboratories using the same reference protocols for DNA extraction and PCR1. Regardless of their random and systematic errors, we observed a high consistency in the ecological assessments (IBD and IPS scores) for both experiments. While most participants developed their own protocols, they agreed to perform “blind” experiments with an agreed standard laboratory protocol. Thus, our proficiency test results demonstrate the potential transferability of a common DNA extraction and PCR amplification protocol, as reproducible results were obtained across different participants. Diatom molecular taxonomic assemblages also exhibited high consistency amongst participants for the three calibrated samples, despite an unequal detection of taxa of low abundance (< 1%). For ecological assessment purposes, this is acceptable as IPS and IBD scores (and, indeed, most other metrics in common use) are primarily driven by taxa with proportions exceeding 5% (
The coefficient of variation (CV) values underscores the stability of most metrics, except for the DNA concentration during the E1 experiment, which exhibited high variability. Variation in DNA concentration from the same sample and protocol within a participating laboratory may arise due to technical factors, such as pipetting inconsistencies and equipment variations, sample heterogeneity, potential contamination and inherent biological variability in cell abundance or extracellular DNA content (
The high degree of similarity observed amongst participants and the reference laboratory in diatom assemblage composition when using identical DNA extraction and PCR amplification protocols (E1 and E2) attests to the reliability and consistency of these protocols, reinforcing their reproducibility. This suggests that standardised protocols can form a foundation for inter-laboratory consistency in diatom analysis. Despite the controlled conditions of the proficiency tests, with calibrated samples, identical protocols and reagents, not all sources of variability could be evaluated (operator, machines, participant lab environment etc.). Source of variability within eDNA studies can be highly diverse and, thus, difficult to predict as environmental characteristics might change from one waterbody to another, affecting eDNA approaches (
Factors affecting test reproducibility amongst laboratories can be related to laboratory environment, biases introduced by human technical practices, model of instrument and calibration (
Finally, the proficiency test we conducted was focused on evaluating DNA extraction and PCR amplification protocols. While these steps are crucial and our results validate the potential to transfer laboratory protocols, it is important to acknowledge that there are other key stages in the metabarcoding workflow that are known to introduce variability. These include library preparation methods (
The results of our proficiency experiments E1 and E2 showed that the implementation of a unique standardised molecular biology protocol can be easily achieved. Although this “One for all” strategy would favour the harmonisation of diatom assessments within freshwater monitoring programmes at the European scale as only one protocol would be used, it is not recommended for several sociological, economic and political aspects (
Variations in the relative proportions of certain diatom taxa depending on the extraction protocol were previously highlighted by
The variability in taxonomic lists introduced by different PCR amplification protocols was surprisingly low in E4, considering the known biases associated with this technique. Our objective was to evaluate the potential effect of different PCR amplification protocols on diatom IBD and IPS indices and not to decipher all the variability within ASV or taxonomic data. Further analysis is needed to determine whether the observed biases are attributable to the specific Taq polymerase or thermal cycler used, as these factors may also contribute to variability (
Results of our inter-comparison E3 and E4 experiments showed that there is a clear influence of participants and methods on the overall diatom assemblage structure, although the general composition remains consistent. Despite significant variations in IPS scores amongst participants, the observed range of variability falls within acceptable limits compared to typical variability observed in morphological intercalibration studies (
The morphological approach remains the standard method for diatom identification in monitoring applications such as the WFD. However, this method requires significant time and expertise from highly-trained analysts and can result in significant variation in metric scores between operators (
Morphological intercalibration exercises are already being performed in Europe (
If intercalibration and inter-comparison exercises are essential parts of routine monitoring to maintain data continuity and consistency between future routine operators, clear guidelines need to be defined to coordinate the integration of molecular and morphological approaches at a national scale. Several documents propose practical guides on the use of eDNA approaches for freshwater monitoring (e.g.
If results from our intercalibration and inter-comparison experiments is to contribute to the development of standards, a first priority would be to decide how to manage the integration of diatom quantification into diatom indices. As an example, several strategies were proposed to handle diatom quantification through metabarcoding using species correction, based on diatom cell biovolume (
These intercalibration (proficiency test) and inter-comparison experiments demonstrate that diatom DNA metabarcoding offers a highly reproducible ecological assessment if the main taxa are included in the reference sequence database. In the experiments, participants used both a standard and an own-choice protocol and performed DNA extractions and PCR on calibrated samples. Congruent results (similar composition and indices) were obtained between laboratories, revealing: (i) the robustness of DNA extraction and PCR protocols between laboratories and (ii) the possibility of using different protocols (i.e. with different reagents), adding more variability to the assemblage composition, but without affecting the outcome of ecological assessment of the samples. As a result, we propose the “all for one, but prove them all” concept, suggesting that many different molecular approaches can be used as long as their functionality can be validated with the approach provided here (calibrated samples, negative and positive controls) and if key minimum steps (to be defined for future standards) are followed. Thus, our study shows how diatom metabarcoding can be included into routine monitoring of freshwaters in a regulatory context.
We acknowledge the contribution provided by the Biodiversa+ DNAquaIMG project, the Office Français de la Biodiversité (OFB), the GeT-PlaGe plateform and the CERCA programme of Generalitat of Catalonia and the Slovenian Research Agency (P1-0245, J2-4428).
The views expressed in this article are those of the author(s) and do not necessarily represent the views or policies of the U.S. Environmental Protection Agency. Mention of trade names or commercial products does not constitute endorsement or recommendation for use.
The authors have declared that no competing interests exist.
No ethical statement was reported.
This work was funded by the European Cooperation in Science and Technology (COST) Action DNAqua-Net (CA15219), Macherey-Nagel, Takara, ID-Gene, SCIMABIO Interface, INRAE (French National Research Institute for Agriculture, Food and Environment), the Cyprus University of Technology, WAT-DIMON (EUROSTARS/0519/0005), INRAE Transfert, the Eco-AlpsWater Interreg Alpine space programme, The Swedish Environmental Protection Agency and The Swedish Agency for Marine and Water Management, the Croatian Science Foundation (UIP-2014-09-6563 and UIP-2020-02-7868).
VV, AnB, AgB, CeC, ET, MK, MGK, MP, PP, FR, ST, RS, MIV contributed to the conceptualisation of the overall study. VV, AgB, CeC, PP, FR, MIV designed the methodology of the study. By participating to the lab experiments, VV, KBA, LAPG, BB, AnB, KKB, JB, CaC, CeC, JNM, FM, MMD, AM, JPB, PP, JP, KT, ST, RS, MV, DW, JW contributed to the investigation of the study. VV, BP and SFR performed the formal analyses of the data. VV, BP, SFR and AgB wrote the original draft. VV, BP, SFPA, KBA, CaC, ET, MJ, MK, MGK, EP, PP, FR, ST, RS, MIV and JZ participated to the reviewing and editing of the draft. VV was in charge of the supervision of the research activity planning and execution. VV, EA, SFPA, AgB, CaC, MD, ET, MK, MGK, MMD, JP, MP, EP, KS, KT, ST, RS, MIV and JZ were involved in the management and coordination of the project administration of the research activity planning and execution.
Valentin Vasselon https://orcid.org/0000-0001-5038-7918
Sinziana F. Rivera https://orcid.org/0000-0002-0812-9031
Éva Ács https://orcid.org/0000-0003-1774-157X
Salomé F.P. Almeida https://orcid.org/0000-0001-7240-7967
Karl B. Andree https://orcid.org/0000-0001-6564-0015
Bonnie Bailet https://orcid.org/0000-0001-8784-5687
Ana Baričević https://orcid.org/0000-0002-7082-1977
Kevin K. Beentjes https://orcid.org/0000-0001-7669-3809
Agnès Bouchez https://orcid.org/0000-0001-8802-6966
Camilla Capelli https://orcid.org/0000-0002-4617-305X
Tina Elersek https://orcid.org/0000-0003-3296-3808
Maša Jablonska https://orcid.org/0000-0003-1988-3600
Maria Kahlert https://orcid.org/0000-0001-9643-4281
Jan-Niklas Macher https://orcid.org/0000-0003-3010-7522
Federica Mauri https://orcid.org/0000-0001-8502-8798
Andreia Mortágua https://orcid.org/0000-0003-0182-0513
Jan Pawlowski https://orcid.org/0000-0003-2421-388X
Javier Pérez-Burillo https://orcid.org/0000-0002-8489-2389
Martin Pfannkuchen https://orcid.org/0000-0002-6253-4716
Panayiota Pissaridou https://orcid.org/0000-0003-4841-7676
Frédéric Rimet https://orcid.org/0000-0002-5514-869X
Kálmán Tapolczai https://orcid.org/0000-0003-1453-767X
Susanna Theroux https://orcid.org/0000-0002-9812-7856
Rosa Trobajo https://orcid.org/0000-0001-9498-3797
Marlen I. Vasquez https://orcid.org/0000-0002-9849-5616
Jonathan Warren https://orcid.org/0000-0003-3381-3852
Jonas Zimmermann https://orcid.org/0000-0002-0522-0569
Benoît Paix https://orcid.org/0000-0001-8159-8574
Raw reads were deposited and are publicly available in the NCBI Sequences Read Archive (SRA) under the BioProject accession numbers PRJNA1187555 for experiments E1 and E3 and PRJNA1187576 for E2 and E4. Complementary data are available on Zenodo online repository system using this link: https://zenodo.org/doi/10.5281/zenodo.12708767.
Additional information
Data type: docx
Explanation note: table S1. Morphological diatom taxonomic list for the Lake (L), the River (R) and the Mock community (M) samples. table S2. Summary of methods used by each participant for the experiments E3. table S3. Summary of methods used by each participant for the experiments E4. table S4. Results of the PERMANOVA test conducted with the “participant” factor for the E3 experiment. table S5. Results of the PERMANOVA test conducted with the “Participant” factor for the E4 experiment. table S6. Results of the PERMANOVA test conducted with the “Enzymatical lysis” factor for the E3 experiment. fig. S1. Instructions given to the participants for the experiment E1. fig. S2. Instructions given to the participants for the experiment E2. fig. S3. Instructions given to the participants for the experiment E3. fig. S4. Instructions given to the participants for the experiment E4. fig. S5. Community composition and NMDS plots performed with the Bray-Curtis dissimilarity index calculated with the datasets obtained from E3 experiment, with the mock, river and lake samples.