Meta published data2vec 2.0, a self-supervised learning algorithm that trains 16x faster than masked autoencoders on ImageNet-1K vision tasks while achieving equivalent accuracy, 11x faster than wav2vec 2.0 on speech, and 2x faster than BERT-equivalent on NLP benchmarks. The algorithm learns across vision, speech, and text using a single unified approach by predicting contextualized representations rather than raw pixels or tokens, and uses masked training and efficient decoder architectures to reduce compute. For financial institutions, this demonstrates that training-efficiency gains in foundational model development are being achieved through algorithmic innovation beyond scale, with implications for the per-unit cost of capability and capital requirements to reach frontier performance.
84 other tracked developments touching the same claims.