Finance ministers, central bankers and financiers have expressed serious concerns about a powerful new AI model they fear could undermine the security of financial systems.
The development of the Claude Mythos model by Anthropic has led to crisis meetings, after it found vulnerabilities in many major operating systems.
Experts say it potentially has an unprecedented ability to identify and exploit cyber-security weaknesses - though others caution further testing is needed to properly understand its capabilities.
Canadian Finance Minister François-Philippe Champagne told the BBC that Mythos had been discussed extensively at the International Monetary Fund (IMF) meeting in Washington DC this week.
“Certainly it is serious enough to warrant the attention of all the finance ministers,” he said.
I have no inside knowledge on this particular work, but their previous work on the OSS-fuzz targets and on Firefox were all excellent quality bug reports.
Seriously. Look them up.
They were all reproducible ways to trigger faults in ASan builds. That’s by definition memory corruption. We can argue about whether all of them are exploitable, but a) they need to get fixed regardless b) we know that even tiny memory corruptions can often be leveraged into a compromise given enough effort.
The question isnt how good the results are, its whether you can achieve the same quality for the money without an llm.
The stories of the bsd bug say they spent $20k on compute alone (and who knows if thats before or after VC subsidies). Then they had so many reports they need to pay some of the top experts to triage which ones were real.
And the result? no remote code execution, no data theft. A remote crash. Its a real bug that can cause problems but its not actually an exploit.
The sad thing is there really could be something new and useful in ai model security, people are seeing good results by automating the reproduction step, but the presentation of it as too dangerous to release and a massive change just sound like pure marketing.
Most likely its just too expensive to do this unless youre a vc funded op with its own compute and want a pr campaign to stop people thinking about how shit the source code you just accidentally released is.
Yes of course they were. Professional security researchers tend to produce professional, high quality reports.