Serology-based tests have become a key public health element in the COVID-19 pandemic to assess the degree of herd immunity that has been achieved in the population. These tests differ between one another in several ways. Here, we conducted a systematic review and meta-analysis of the diagnostic accuracy of currently available SARS-CoV-2 serological tests, and assessed their real-world performance under scenarios of varying proportion of infected individuals. We included independent studies that specified the antigen used for antibody detection and used quantitative methods. We identified nine independent studies, of which six were based on commercial ELISA or CMIA/CLIA assays, and three on in-house tests. Test sensitivity ranged from 68% to 93% for IgM, from 65% to 100% for IgG, and from 83% to 98% for total antibodies. Random-effects models yielded a summary sensitivity of 82% (95%CI 75-88%) for IgM, and 85% for both IgG (95%CI 73-93%) and total antibodies (95%CI 74-94%). Specificity was very high for most tests, and its pooled estimate was 98% (95%CI 92-100%) for IgM and 99% (95%CI 98-100%) for both IgG and total antibodies. The heterogeneity of sensitivity and specificity across tests was generally high (I2≤50%). In populations with a low prevalence (≤5%) of seroconverted individuals, the positive predictive value would be ≤88% for most assays, except those reporting perfect specificity. Our data suggest that the use of serological tests for large-scale prevalence surveys (or to grant "immunity passports") are currently only justified in hard-hit regions, while they should be used with caution elsewhere.