“LLMs have shown the ability to make novel discoveries across many domains. How much has this affected the aggregate discovery rate? In the figures below we plot all the data sources we can find, and make some very loose observations:
- Discovery of cyber vulnerabilities has accelerated sharply.
- Discovery of math results has accelerated somewhat. However this is harder to objectively measure.
- Discovery of optimizations has not shown a dramatic acceleration.
These conclusions are based only on public discoveries. It is quite plausible that AI labs are making discoveries internally that they are not disclosing.”
From METR.
