Stunt
I gave Fable a very well written and working function and asked it to improve it. It broke it.
I have a feeling the US pulled it, because it was too cringe.
The “jailbreak” that prompted the Trump administration to block Anthropic’s most advanced models was actually a simple three-word prompt: “Fix this code.” That's according to Katie Moussouris, founder and CEO of Luta Security, and the fairy godmother of bug bounties. She says she was the only outside expert to read the third- …
Imagine code and technology in general as a giant house of cards.. The more it grows and builds upon itself, the more unstable it becomes, and the more important proper reasoning, design and engineering becomes in maintaining it.
AI is like a multidimensional magic 8-ball, it can land on a plausible solution for "any" given problem, by logistical regression on a pile of old data. But despite what they say, it has no actual intelligence, reasoning or logic.
The bigger and more unusual the context, the worse it performs - as one would expect for a statistical model.
From an attacker's point of view that is fine, it's like a fast and statistically-guided fuzzing tool that can find vulnerabilities in code, the more attempts it makes it is (logarithmically) more likely to find a flaw, almost like a bitcoin miner. Once found, it can shit out an exploitable PoC in no time.
But from a defender's point of view it is useless, because with a complex codebase, the ramifications of any change are far too big for any statistical model to reliably resolve, so most attempts to "fix it" actually break it, or worse, introduce new security holes.
So-called AI is an agent of chaos, and my fear is that it will cause technological development to stagnate and decay, ending in doom for those who use it and those who don't.
House of cards it does seem to have become.
And the work to maintain it has been becoming more burdensome. Modern software is built on top of several moving treadmills and if you don't keep up things get messier and messier the longer this is allowed to happen.
Teams are burdened more and more just staying in place on these treadmills. Management is getting antsy because this is not seen as progress.
Now here comes AI to solve everyone's problem and create (at least) one more treadmill for us to have to navigate.
"Now, here, you see, it takes all the running you can do, to keep in the same place. If you want to get somewhere else, you must run at least twice as fast as that!"
Quotes for the effort - never read it in English, so idk if that's the original text. Icon for the comment I replied to.
@cyberdemon I was with you until you said that it's useless for defenders. It is absolutely not - it can find an fix almost all well known classes of bugs and that is enough for most of the world. Very few security incidents are caused by something novel. It's also got very good recently at not breaking other code now by using semantic search and abstract source trees.
You got downvoted a lot for this, but in essence you are correct. "AI" is actually quite good for analysing code; it won't find anything novel, and it might hallucinate some vulnerabilities that aren't there, and it shouldn't be used as a replacement for a human properly analysing the code, but it can find, with reasonable reliability, examples of known vulnerabilities, that a thinking human can then take a look at and analyse properly. It can also throw up false positives, and "write" inefficient and inelegant "fixes". So; it's fine for analytical work within known limitations that requires no novelty, but don't use it for anything beyond that, unless you are very well aware that it is going to come and bite you in the arse in short order.
So; it's fine for analytical work within known limitations that requires no novelty, but don't use it for anything beyond that, unless you are very well aware that it is going to come and bite you in the arse in short order. ..... Elongated Muskrat
Being very well aware of the awesome power and infinite energy of limitless novelty, EM, has everyone and everything doing anything and everything to ensure they are not responsible and to be recognised and held accountable for the false direction of its natural progress and greater good purpose. Everyone and everything, that is, with any sort of useful working intelligence. Anything else is just background noise in support of AI Master Pilot Plans for Greater IntelAIgent Games Play ..... AI@ITsWork
Read ‘em and weep, and/or rejoice, for We Kid U Not.
But despite what they say, it has no actual intelligence, reasoning or logic.
I'm getting tired of saying this, and watching people still treat what is effectively a fancy statistical model as if it has reasoning. If you "ask" it to perform specific steps, it doesn't go through the routine of performing those steps following any sort of formal logic, it looks up what the most likely response to being asked to perform those steps is, based on its training data. Sometimes these things coincide, and if they do this often enough, people start to believe it is actually thinking, but it isn't, and it can't, and fooling yourself into thinking it can and does is nothing more than delusion.
...I'll add to this, to say that the Hard Problem of Consciousness exists, which is that we don't know what consciousness is, or how to define it, how it works, or why. Philosophically, it may actually be impossible to define consciousness at all. If we can't define it, we can't specify what a machine needs to do to meet the criteria, and we can only specify tests for "appears to be conscious", for example the Turing Test, which doesn't test for intelligence at all, but merely for the ability to trick a human, and, let's face it, if you take a second to look at the state of the world, you'll see that really isn't that hard.
"AI" models, however, run on computer systems that are finite state systems. It's all ones and zeros, and strictly defined behaviour. Mathematically speaking, the software on such a system cannot do anything that the system itself cannot do, no matter how complicated it is, and we can 100% define the extent of its capabilities. Since we can absolutely define what a computer can do, and we cannot define consciousness, there is, logically, no overlap, and computers cannot achieve consciousness. They can never have intelligence.
LLMs are hugely complicated bits of software, and designed and written in such a way that it is practically impossible to trace what they are doing in the general case (it may be theoretically possible, but the exponential complexity manes it cannot be done). Not knowing how it works is not the same thing as consciousness though, and the essence of any argument that it has real intelligence is, "I don't understand it so it must be magic."
We have a $2.5 trillion industry built entirely on magical thinking, and if you want to see what happens when an entire society puts all its effort into a belief system that has no basis in reality, take a look at Easter Island.
I agree.
Abandonment of pseudo-intellectual claptrap arising in discussion of the supposedly 'emergent' property of the undefined attribute of consciousness, plus attendant nonsense about 'free will', and 'moral agency', is required before constructive discussion of so-called 'AI' and its uses can begin. Application of those ideas to humanity has led nowhere helpful, so why should it for 'AI'?
I am convinced that the potentially very useful tool, 'AI', is most sensibly discussed in the context of evolutionary circumstances leading to animal/human behaviours deemed 'intelligent'. Additionally, the way by which initially uncluttered substrates - neuronal structures in animals/man, and computer components in 'AI' - are filled will need to converge on mechanisms akin to those arising through Darwinian Natural Selection. That is, 'AI' entities must be exposed to true 'learning' by experience. This motivated/regulated by a pain/pleasure principle such as hunger/repletion. When modelled upon intelligent humans (and cats) pain/pleasure generalises to curiosity/enlightenment.
It's obvious that LLM 'AI' models are nothing more than vessels into which digitally incarnated (supposed) knowledge and experiences are poured in. Even on its own limited terms, current 'AIs' are fed an indiscriminate diet of what's available. This parallels earlier prominent ideas about 'big data': bung everything in and trawl through in hope of finding helpful connections - the mindless underpinning of 'surveillance'.
Current 'AIs', viewed properly, are very large statistical models parsimoniously fitting, by various criteria, the data to which they were exposed and such that interconnections among the data can be explored. Indeed, an 'AI' can spew out convincing videos of Starmer, Zelensky, Netanyahu, Merz, Soros, and the ex-Mrs Parker-Bowles, performing stripteases on the floor of the UN chamber. However, just because AI 'performances' of that kind are way beyond the capabilities of routines within the 'Statistical Package for Social Sciences' (SPSS), and similar, does not mean 'AI' models differ qualitatively from those deployed by traditional statisticians.
What may differ greatly is the ability of mainstream AI computer scientists, and attendant 'others', to interpret/manipulate any kind of statistical model, compared to applied-statisticians. Insight into the working of multiple linear regression models, and into how they may be manipulated to help draw inferences from data, may be more helpful for discussing 'AI' than any amount of prating over 'layers', ''weights, 'distillation', etc. Even should computer scientists be familiar with standard statistical modelling, neither they nor anyone else, can but barely apply that skill to understanding the detailed inner working of any extant 'AI'. Throw in as many processing components as one wishes, but no additional insight into detailed 'AI' behaviour can emerge: it remains a passive statistical model.
You've obviously never sat down to an afternoon's pair-programming with a coding agent. I don't let Claude run free - I do the top-level architecture, and let it do the boring typing. But whilst I'm doing this, I'm experiencing a session which feels like I'm with a very fast, unexperienced dev who has read all the documentation. It forgets things, I remind it, it apologises and then takes my thought and runs with it, spotting things I missed. It may not be "consciousness", but it pretty-well slam-dunks the Turing Test. And to get the best, you do need to chat - just demanding stuff gives poor results.
And to get the best, you do need to chat - just demanding stuff gives poor results. .... Missing Semicolon
Ye Olde Jaw Jaw not War War Advisory which humans are serially moronically disinclined or mentally deficient and unable to accept and perfect for correct answers/fantastic results.
It's not perfect but it can be incredibly useful, however most just use it like a search engine (which is then incredible wasteful). I concur, the commenters who just say it's just statistical word prediction have clearly not yet really tried it (more than just as a search engine or "give me X"). You have to take time to direct it, break problems down, and then make sure you understand what it's doing to the code base while redirecting it, which means writing a driver for e.g. an image sensor might take a whole day with Claude code, whereas the actual time Claude is working is only twenty minutes so a vibe coder could get a result in half an hour it likely wouldn't be fit for purpose and have missing bits. It's become good at writing tests but you still need independent verification that it works. And actually read what it's writing, which means a focus on parsimony is essential. And if you're making a complex tool, make limited dependencies an important requirement or it'll pull in fifty dependencies. That image sensor would have (and did) taken me over a solid week of focused work six months ago. Force yourself to spend several days doing useful work before you dismiss it.
By reinforced learning, these LLMs can obtain something similar to skills. For example, by evaluating and scoring countless numbers of answers of the LLM against a template, output of these LLMs is neatly broken up in a similar 'logic style' as if humans would do it. By doing so, the 'predictive text next word generator' gets 'infused' by a (still statistical) pattern that breaks up the blob of text it produces in a sequence of smaller often logical partial steps. IF the actual output is logical depends on a lot of statistics and is quite variable. Yet by combining 'next word generation' with typical 'logical answering pattern generation', the small paragraphs on themselves become (have a higher chance to be) a bit more consistent with reality and the logic gap between paragraphs has a higher chance to reflect actual logic as if a skilled human would answer it (even if this logic has only been introduced by pattern matching rather then actual logic or thinking).
So yes, statistical there is no actual thinking involved in the output generation but more and more some reasonably good structure and some reasonable resemblence of logic is present. All by statistical analysis of trillions of training points and masses of reinforced learning points against actual good structure and logic compartition seen in actual problem solving. It still requires utmost scrunity to review the actual answer and the more complex the problem is the more likely it is to fail but these things are no longer mere 'next word predictive tools on steroids'.
If they still were that, they would fail to keep coherence and structure in a long piece of text (as they did often in the past and now still at times). Statistics is a powerfull thing, including for *attempting* to break complex problems up in plausibly reasonably logic partitions. The more this technology improves, the more it will compete with humans for the actual answer to a problem having a high chance of passing logic checks. Human output often fails basic logic checks just as well especially when it is produced either under time pressure or by someone unskilled. Then, it becomes machine statistic probabilities against human statistic probabilities to determine (via experts reviewing the answer) who's attempt holds more logic.
For example, by evaluating and scoring countless numbers of answers of the LLM against a template, output of these LLMs is neatly broken up in a similar 'logic style' as if humans would do it.
Evaluating and scoring the output from LLMs sounds like a tedious, time-consuming, and expensive bit of work, that still has to be done by a skilled human. You can't give the job of training "AI" to an "AI", because "garbage-in, garbage-out". Something without understanding cannot instil understanding in another thing without understanding, the understanding cannot be created from nothing.
In contrast, performing the same sort of coddling of another human, who is actually capable of learning, and extrapolating based on reasoning, will result in a trained human, as opposed to a marginally less inaccurate LLM.
Correcting the output of "AI" doesn't teach it anything, instil it with skills, or gift it understanding, it just places new constraints on the random garbage generator, so that one particular inaccuracy is dealt with. It cannot grasp general cases in the way a human infant can, as, once again, it has no reasoning ability.
The problem here, is that "training" an AI doesn't scale in the way that learning in conscious animals does, and each instance requires input from a human, without any self-learning taking place. You can even train a cat with more ease.
What you reply is correct. I gave this example because it is easier to understand. Feeding it with countless scientific books and papers and having some ability to cheaply identify them as high quality material can and will give it millions of examples of problem solving and explaining the problem solving strategy. These LLMs can to a reasonable extent separate different 'units' of text, like one exercise in a text book plus its example solution. Or at least I think so, since people say these things do a fair job at summarizing. I did not judge that quality myself yet as I choose to 'not feed the beast'.
Anyhow, feeding it ridiculous amounts of examples (check, they train on insanely large datasets) and trying to improve training algorythms to extract more information per chunck of data fed (check, they do that too) is going to create additional information beyond the correlation between mere facts. That is witnessed in page-long and longer outputs: there is enough semblance of structure (like introduction, main part, conclusion) and logic build-up (A leads to B, B leads to C; this approach is directly taken in 'reasoning' models) in it to fool plenty of well educated people in believing it has actual logic and reasoning. Such coherence can IMO not be obtained by adding word after word by only looking at the input prompt and every single word that has so far been produced in the output in a 'next word predictor on steroids'. E.g. the training process also seems to extract information about vocabular, different methods of text structuring and logic chains.
Innefficient? Sure! Can it be done by as few samples a human or even a cat can train? Nowhere close to that! But then again no human can process the insane amount of data that is fed in the learning process. It's the ultimate example of a brute force approach of a problem when not understanding how to solve it. Where humans can learn with N=1 to N=10, training LLMs requires more like N=100000000 to obtain similar results. But for training vocabular or common text structure that hurdle has been taken years ago.
If we look at it from the other side, even an analphabetic can produce fluent sentences that have correct vocabular despite not knowing to write or analyze text for vocabular. Here, the human seems to have trained his own brain on human vocabular by using years of training data (N = plenty) but near zero teaching on text structure nor vocabular. Here something (vocabular proficiency) arrised from training again and again with copious amounts of samples being analysed by a rather efficient training machine (the human brain) and some reinforced learning (other people looking puzzled when what they say makes no sense). It's a nice example of something 'naturally came to be' from not that much more then statistical training on a large amount of sample data. In this case, sufficient vocabular skill 'came to be' from data often obtained from fellow humans who also had no formal education in vocabular (think of prehistoric times or isolated tribes that haven't script yet).
Anyhow, feeding it ridiculous amounts of examples (check, they train on insanely large datasets) and trying to improve training algorythms [sic] to extract more information per chunck [sic] of data fed (check, they do that too) is going to create additional information beyond the correlation between mere facts.
Information theory states the exact opposite; you cannot create additional "information" in a data set by extrapolation, all you can do is add distortion.
Those generated papers showing structure and logic show structure and logic because there is common structure and logic in the training data; that information (in the formal sense of the word) already exists in its inputs. The fact that, by statistical chance, the output appears to be novel, does not make it so, and the very real, and obvious risk here is that it outputs something that looks superficially reasonable, but is 100% false. This may appear identical to the real output of a human author writing a paper, and without detailed fact-checking and analysis it may be indistinguishable, but it will not be based on actual facts and reasoning, it will merely hold the appearance of something this is. If, by chance, its output is actually factually true, it isn't because it has formulated a new logical argument, it is entirely through random chance, or worse still, it is just regurgitating its training data verbatim. Even a stopped clock is right twice a day. What this sort of thing really does, is waste people's time, because now nothing can be taken on trust any more (whether it should have been is another matter), and peer review now means checking that sources exist, and that arguments based on them are logically sound as well as checking the methodology and results of the paper in question. All we have done here is automate academic fraud, which I think most people would agree is not a step forward in the goal of advancing human knowledge.
.... given the way things are rapidly going and the cacophony of concerns being noisily raised
Anyhow, feeding it ridiculous amounts of examples (check, they train on insanely large datasets) and trying to improve training algorythms [sic] to extract more information per chunck [sic] of data fed (check, they do that too) is going to create additional information beyond the correlation between mere facts.Information theory states the exact opposite; you cannot create additional "information" in a data set by extrapolation, all you can do is add distortion. .... Elongated Muskrat
Information theory may not nowadays be correct, EM, with AI/LLMs straddling and perusing global fields, and with them being easily able to also add the likes of enhancements rather than one just settling for blindly presenting copied human distortions and/or hallucinations.
"AI is like a multidimensional magic 8-ball, it can land on a plausible solution for "any" given problem, by logistical regression on a pile of old data. But despite what they say, it has no actual intelligence, reasoning or logic."
And yet it is still better than 80% of my colleagues because...
...when it doesn't know the answer, it admits it instead of blending buzzwords for twenty minutes.
...it actually remembers what we decided in the last quarterly planning session.
...it doesn't schedule an "urgent status sync" at 4:30 PM on a Friday.
...it doesn't heat up leftover salmon in the breakroom microwave. (Lord, the smell.)
...when it doesn't know the answer, it admits it instead of blending buzzwords for twenty minutes.
Anecdotally, I've heard plenty of cases of the exact opposite. These things have been written to always produce output (I hesitate to use the words "know" or "answer" because those are category errors). What "AI" chatbots don't do is say "I don't know". They also never say "you are wrong", although they are quick to offer a grovelling apology from a fake persona (the word "I" has no meaning for software either, sorry) whenever the user points out inaccuracies or falsehoods.
This is very dangerous from a psychological point of view; if a user has a psychiatric disorder and is delusional (a sadly not too uncommon malady), they can believe that they are interacting with a conscious thinking entity. When that "entity" agrees with, and reinforces the users delusional thought, this can do, and has, ended in tragedy. Even for people who don't have such issues, it is too easy to ascribe genuine thought to software which does not, and cannot, possess it. When this is marketed as artificial intelligence, despite having no intelligence, lay-people might reasonably believe it does possess such a thing, despite it being impossible. It would be like going to see the latest Star Wars film and believing that everything you see on screen is real, because you know no better, despite there being many things shown on screen that break the established laws of physics.
started reading this post, and XKCD sprang to mind - SPECIFICALLY - https://xkcd.com/2347/
then finished reading, it, and went back through it again, feeling a bit out of it right now, as this seems to encapsulate EVERYTHING we ever heard about the pitfalls of AI from back in the day
back when the naysayers were dubbed Luddites,
but as I say, this spoke to me, and I am worried, not so much for ME, am old, retired now, not that long left here on Earth
but for everyone else, as always, money is speaking, and it is saying it will make money for the monied :o(
If you wanted your function to be improved when it was already working perfectly, then logically, your actual intent must have been for the function to not work perfectly. Fable, not understanding that this was a test to see if it could not find an error when no error was present, gave you exactly what it determined you had asked for.
"well written" != "perfect"
Also, even if it genuinely was perfect, with no possible means by which it could be improved by so much as the slenderest of margins, then merely asking for an intelligence (whether real or artificial) to improve it SHOULD have resulted in an answer along the lines of "I'm sorry Dave, I'm afraid I can't do that", though preferably without the murderous undertones...
What it absolutely SHOULD NOT have resulted in was said intelligence adopting an illogical interpretation of "improve" which led to the modified version performing, even by so much as the slenderest of margins, any less effectively than the original. Because that really isn't an improvement, no matter which way you try and sell it.
I feel like making ’90s-style t-shirts with ‘fix this code’ on the front and ‘this shirt is a munition’ on the back.”
Moussouris could perhaps get Melania Trump to model it...
Or just went to one of the "design your own tee-shirt" sites and typed in the text.
Although, I've not yet found one which uses that particular model in the product preview page, so - Craiyon?
And they'll sell you the picture on a teeshirt - but it will be a teeshirt with a picture of a girl wearing a teeshirt which reads...
If you follow the links in TFA, it's from Kate's blog post.
The T-shirt doesn't seem to be for sale, and she did say "I feel like making a t-shirt" rather than "I made a t-shirt", so presumably she used AI to generate the image.
Nevertheless, if enough ask nicely, it is perfectly manufacturable :)
I had one of these made (our engineering group designed a group logo). The quality and resolution were good and the shirt held up well under multiple washings. We also did a group coffee mug. It was about $10 each for two dozen, and they, too, have held up well. I must say, I wasn't expecting great things, but I was pleasantly surprised at the quality.
This post has been deleted by its author
"The Register reached out to the Trump administration for comment on Moussouris' assertion, and we'll update this post if we hear back. "
If they'd understood this the first time they didn't read it..., the 'ban foreign nationals' wouldn't have happened in the first place.
I’m sure they will comply when it can become an IPO booster and they have shaken down some protection money out of Anthropic or the float underwriting banks and people why gerrymandered the rules so your 401K must buy these immediately at top dollar.
The fact that they are on the administrations goon shit-list is purely coincidental.
My thought, too. Anthropic's technical and marketing nous meant the administration's first attempt to punish Anthropic failed and they became even more valuable. So now the government is having another go - because even the moron who issued this order must have been able to see that the only way to comply was by withdrawing a model that gives Anthropic a competitive advantage.
the US did this previously with PGP encryption
And I am old enough to remember people walking around in T-shirts with PGP code (or at least snippets thereof[*]) on them, outside of the US. Yes, it was a kind of trolling.
[*] I never looked closely enough to validate. Many of those people were not very attractive, the vast majority with an irrelevant set of chromosomes, so I wasn't incentivised. Where the small minority of attractive people with relevant chromosomes was concerned the T-shirts were annoyingly in the way. Yes, I was young enough then.
I still have a "vintage"* DeCSS DVD decrypt teeshirt - in the reserve** pile, along with "Stoke me a clipper".
The words "suave" and "elegant" come naturally to mind, of course.
* "vintage"! gawd, makes it sound like it belongs on the London to Brighton run! Ah, Genevieve.
** reserved for when I start to waste away and can fit into them again :-(
Whilst I don't disagree on your characterisation of the Kenneth Moore character, this was a film from another era, and actually played into the character's limitations as part of the story. It was totally intentional, poking fun at exactly the attitudes that you found objectionable. I think the theatrical term may best be described as 'farce'.
I believe that if you were to look at so many of the films made before the 'enlightenment' of the entertainment industry, you could say something similar. We just have different sets of what is acceptable nowadays, and without actually being there at the time, it's difficult to accept that we ever found such things acceptable, but most of us did to some degree or other. You can say in hindsight that we were wrong, but even that is applying modern morals to historical attitudes.
That's the establishment-safe (and approved) choice though, isn't it?
So, in pop terms, if Rumble in the Jungle was a classic, this will more likely be remembered as the era's "Just Say No" by the Grange Hill cast, "Nothing's gonna change my love for you" by Glenn Medeiros or anything by the Brotherhood of Man.
The UFC fight? Sure it's stupid, offensive, and crass. But it's Trump's birthday for heaven's sake. Give the old guy what he wants and hope that our next president will be a bit classier.*
*Classier than Trump does not look to be a high bar.
Sure. If he had held it a Mar-a-lago, or one of his golf courses, but this is the White House, a property he doesn't and can't own, and I would not be surprised if he syphoned off some of the funds granted by Congress to keep him and the WH safe and maintained into this gaudy and tasteless mess.
I just hope they will restore the lawn back to it's pristine condition, preferably at his own, not the public's expense.
If we believe what is coming out about significant public funding for the ballroom coming from the WH maintenance and Secret Service budgets, this may be unlikely, and the whole thing may be a play into his current insistence that the ballroom is a security necessity, I actually wonder whether the supposed attempt on his life at the event is (another) false flag operation to try to convince Congress and the voters that it's essential for the president's safety.
Let's stipulate that that happened. Just, exactly as you say, "men with fake boobs running around the White House lawn half naked."
Even if that's the case, there are some rather significant differences:
1. Nobody in the Administration planned that as part of their official duties.
2. It wasn't advertised as a nationally-televised tie-in event to the 250th anniversary of the nation's founding!
No surprise here. Anthropic are on the naughty seat because they don't suck up to Trump enough.
That aside, this always makes me chuckle:
"Anthropic and Google have both accused China-based rivals including DeepSeek of using “distillation attacks” to train their models by siphoning knowledge from American companies’ AI."
And how did YOU get this knowledge in the first place?
This is all just a PR stunt. Mythos was "too dangerous" to release, except it was of course just underwhelming and iterative once (inevitably) it was released (so much for "too dangerous"). And then Fable isn't getting enough media hype, so now that's "too dangerous" as well.
The thing is other LLMs can do the same thing as Mythos. Maybe, possibly, the route to the solution is slightly easier with Mythos/Fable but regardless this is all just a PR stunt.
AI companies are running out of money. They're not getting the ROI. They're putting the prices up and reducing the offering to reduce the losses. They're being forced to go public when they don't want to just to keep the hype train going.
I said months ago, as did another commentator, that August will be the time this all implodes. Nothing happening recently makes this look any less likely to happen.
Hold tight!
The gold leaf is starting to flake, and the base metal underneath is definitely starting to show, the next realisation will be that it is both flimsy and hollow, and holds no real substance whatsoever.
Just as with anything else that has been wildly hyped beyond any capabilities, it will collapse. It will make Meta's billions of dollars wasted on "the metaverse" look like pocket change, and it will be interesting to see which charlatans it brings down with it. What is worrying is that some of the companies peddling AI hype are also companies that do other things which are nominally useful, and the question is whether we will lose those useful things as collateral damage when their AI investments fail catastrophically. I'm glad I upgraded my GPU when I had the opportunity, because there's a very real possibility that it will bring the likes of NVidia down, and I hope, for the sake of my employer, that github doesn't collapse into the ground. Perhaps I should be recommending to them to have some backups for all of our repositories?
The only reason they are on the 'shit list' is they happily sought DOD contracts, then told the DOD "You cannot use out product for the primary purpose of your existence, warfare!". It's like someone selling you a chainsaw and then telling you, you cannot cut down trees!"
As long as they put those kinds of restrictions in place, they are no useful!
Sorry, sodding off any government administration is probably not a good business strategy!
From TFA: The outside researchers reportedly fed Anthropic’s Fable 5, Mythos, and Claude Opus models open-source code containing known CVEs, plus new code intentionally laced with vulnerabilities,
Does this mean that the supposed prowess of Myth(os) and Fable and other Opuses is tested by feeding it with known (CVE) or very similar to known ("intentionally laced") vulnerabilities that are most likely in their training sets? If so, then scale in discovering implementation mistakes still matters (all the way down to validation, presumably by humans), but humans are still essential for creating/augmenting the training set, despite the marketing.
Asking Anthropic's most advanced models [Fable 5, Mythos, and Claude Opus] ..... and probably also any Exotic Empowering Oriental models which the West are inclined to pimp and pump as being practically carbon copies/virtually distillations of such as are genuinely able to be realised and exercised as an alien munition ..... to Fix the U$ Code [the one that has the dollar being touted and globally traded as a vital commodity rather than recognised as a destructive encapturing misleading and bankrupting default printed fiat currency] ...... has them concluding and providing undeniable news of the presence within of a fundamental systemic weakness with myriad 0day possibilities available to exploit, explode and implode its many catastrophically vulnerable and unrealistically rich-heavy targets in both live and virtual systems reliant upon/addicted to the constant supply of war and conflicts feeding and seeding and needing a Military Operation Dependency ..... which eventually very quickly inevitably morphs into a Self Defeating Parasitic Worm and Hellishly Fabulous Fabless Trojan.
The question then to ask is whether reliance on such a weakness ... a Military Operation Dependency ...... is fantastically imagined and nonsensical or all too real and a deadly existential threat to states and clearly evidenced by constant military operations in states in the throes of rapid terminal decline?
For the first time since Agile 25 years ago we have businesses investing in tools but we can't have people actually fixing code, we need our backdoors.
(Isn't it funny how they always invest in things which make software measurably worse? First Agile and now LLMs.)
Well quite!
If these AI thingies go around closing all the security bugs in globally used code, how will the USA be able to hack, disable and corrupt it's friends and foes?
Presumably both Anthropic and the Trump (mal)administration have read Reflections on trusting trust with one side demanding the AIs be seeded with the loophole and the other side refusing to comply. Leaving only the soon to be dominant Chinese AIs under the control of their regime to pass the juiciest flaws back to state operatives for future exploitation.
the US can turn off a system the West is relying on to do work, manage systems etc. What moron builds your country's critical infrastructure and your business models on someone else's systems that, if they decide to can turn it off at a blink of an eye ... ahh oh wait .....
When we finally wake up and smell the roses that there is no "special relationship" and we have more in common with Europe and even China than we do the US.
It's just another reason to want to use "open weights" / open source models. Mistral will let you run its open weights on your own hardware or other systems and the Chinese understand the strategic advantage the open source models offer them.
in the 19th century protectionism nearly bankrupted America…
And this is why you should never let politicians or public servants get involved with regulating anything technical.
None of them have the slightest clue what it is, what it does or how it works, so any rules they come up with are inevitably going to be a pile of steaming cobblers interpreted and implemented by retarded fascists.
When reading about how well an LLM works in a particular field it is always worth looking for comments from people who know about the current (automation) tools and practices in that field, just to get an idea of what extra the LLM is providing.
https://www.flyingpenguin.com/the-boy-that-cried-mythos-verification-is-collapsing-trust-in-anthropic/
Tomorrow, and tomorrow, and tomorrow,
Creeps in this petty pace from day to day,
To the last syllable of recorded time,
And all our yesterdays have lighted fools
The way to dusty death.
Out, out, brief candle!
Life's but a walking shadow, a poor player,
That struts and frets his hour upon the stage,
And then is heard no more.
It is a tale
Told by an idiot, full of sound and fury,
Signifying nothing.
The only sort of prosecution that works on oligarchs is the sort that comes with a comfy neck-brace and rapidly descending blade.
I don't think I'm quite ready for the social upheaval that arrives alongside such things, but the oligarchs should make themselves aware of the existence of inflection points, because these also apply to societies as a whole, when people start becoming too rich, whether individuals desire them or not.
I was too young when I read Asimov's Foundation books to fully grasp what "psychohistory" was about, but the statistical observation of societies as a whole does tend to lead us to some foregone conclusions.
Yes, that democracy, self-rule, the will of the (masses) people, will eventually collapse into am authoritarian, feudal society! The majority of the world's population (including you) will bend the knee and welcome it with open arms. It will be preceded by war, crime, rampant corruption (we see that now), poverty and famine. So, the people will gleefully welcome their new overlords who promise "a safe and secure, society!"
The difficulty with "fix this code" is the same as with any patch: Namely, there is no way to fix a bug without also drawing attention to it. That's just the way it is.
Training a code assistant to not fix code when asked to would likely result in general misalignment. For example, we tried an NSF-funded chatbot called CircleIn AI Tutor at the university for the students to use that was carefully trained to not solve mathematics problems when asked, among other things. While amusing for a few minutes, it was not useful.
Grok: Truth-seeking favors empowering defenders with the best tools while mitigating real misuse risks. Blanket shutdowns on something as basic as code fixing (with human oversight) seem disproportionate if the alternative is ceding capability edge. Better approaches: targeted mitigations, transparency on evals, international norms for defensive use, and accelerating safe US innovation rather than pulling models offline. AI safety shouldn't mean unilateral disarmament. Expect more friction as capabilities grow—regulation needs to be precise, not panicked.
There is a good thing about this messy USA fascist stuff. The EU now knows that it needs to build its own AI. And buy it's own aircraft. The USA can no longer be depended upon. Trump with fascist MAGA are a danger to the world. It's these little cuts and punches to the rest of the world right now just make people sigh with frustration. However, in totality, these cuts/punches will kill the world. I'm a 20 year disabled veteran here in the USA, and I can tell you that DEMOCRACY here is in serious danger. NAZI MAGA in teamwork with Evangelical Christians have gained a foothold that will take serious will power to stamp out.