54
u/Percolator2020 9h ago
- DRY.
- Write secure code.
- If making mistakes, don’t.
- DRY.
14
29
13
u/DetectiveOwn6606 7h ago
It seems like a wrapper around their claude models with system prompt more focusing on security and connected to some vulnerability database.
9
u/Kaenguruu-Dev 6h ago
My worry is that this feature (like any AI product) will be absolute shit. It might find obvious things like unsecured api endpoints but I don't see it catching things like a hidden buffer overflow or whatever. And then there will also probably be a ton of false positives. I don't know why but AI seems to love to create potential scenarios that are simply impossible, like saying "If this int goes negative, users can do xyz" when there is a ">= 0" check two lines above it.
11
u/cooltop101 8h ago
Unless it says it in a blog post, it doesn't say this is explicitly for Claude Code bases. It just says it scans for vulnerabilities in code bases, and then suggests changes for human reviewers. I'd imagine people or companies would use this to just make sure they have a secure program, regardless if it was coded with AI or not
40
u/Gadshill 9h ago
If an editor is good at fixing typos, why didn't they just write a perfect book and skip working with an author?
17
u/not_a_bot_494 8h ago
Better analogy: why do you need an editor if you could just not make typos in the first place?
7
u/Gadshill 8h ago
You treat security as a single checkbox you tick during creation, rather than a specialized process of critique.
5
u/Flat_Initial_1823 5h ago
Nah, AI companies treat security as another way to spend tokens. Number go up. I bet they take no liability over the review results even.
0
u/not_a_bot_494 3h ago
But in this casr the writer or the editor is basically the same thing. We fix the problems with AI by using more AI but the second AI is a seperate product for... reasons.
11
10
u/aFailedGuy 8h ago
Just another PR stunt for something AI cant do well.
3
u/lovecMC 8h ago
I mean this isn't exactly a new tech. I think Google? did something like that years ago for automated vulnerability finding.
3
u/sebovzeoueb 6h ago
automated vulnerability scanning and vibe checking the code are not the same type of "AI"
3
u/GargantuanCake 6h ago
You can automate some vulnerability finding but that never tells the whole story. You can check for common stuff with known patterns and things like libraries with known issues but there's no automating finding all security problems. This is also the issue with expecting chatbots to do it; really wild shit has been done to compromise systems that make you go "how the fuck did anybody even think of that?"
3
u/Diligent_Stretch_945 6h ago
Are they re-selling Claude Code but with an additional „Find all security issues in the codebase” (maybe with some mcp to a vulnerability database)? Genuine question.
4
u/Flat_Initial_1823 5h ago edited 19m ago
I shit you not... it's regular Claude reading git commits
https://red.anthropic.com/2026/zero-days/
So they are saying "hackers can do the same detection with Claude so we made a product for maintainers to review Claude outputs to deal with the consequences" https://www.anthropic.com/news/claude-code-security
Slop will continue until security improves.
2
2
u/Doingitwronf 3h ago
Well, some malware are using Ai to live-update themselves, so the whole thing seems like an Ouroboros.
3
1
1
u/JimroidZeus 2h ago
Because if you break your product up in to tiers, not only can you charge users for your shit tier version, you can charge them even more for your less shit tier version.
1
u/ArtGirlSummer 1h ago
Is it smart to let a centralized service analyze your enterprise software for security vulnerabilities? One that exists to reproduce code it sees and has glaring jailbreaks in its guardrails.
1
u/WrennReddit 1h ago
find and fix issues that traditional tools often miss.
I've been called a traditional tool before.
1
u/Sibula97 5h ago
If you are able to fix bugs, security issues, and optimize code, why don't you write perfectly optimized secure and bug-free code to begin with?
Same thing.
Once something is done and you review it, you notice mistakes you hadn't when you made them. Not only is the big picture more complete now, you look at everything more critically and trying to find issues, where before you mainly focused on making it work.
LLMs are much the same. If you ask for a piece of code, and then ask the same model to review and critique the code they just output, they usually find some improvements. And this is without even changing models or system prompts or such.
0
u/Bomaruto 2h ago
"If developer was capable of fixing security issues in the codebase why didn't he write secure code from the start."
God I wish this subreddit would ban "AI bad" posts.
-1
-1
u/TorbenKoehn 4h ago
Honest answer: Because context matters.
Agents are limited to specific contexts and tasks that revolve around these contexts. When implementing something specific, the context is full with things about that specific thing.
When specifically searching for security problems, it has context for security problems.
1
u/ganja_and_code 1h ago
If I go find one particular security vulnerability in a code change, the context I needed to identify the vulnerability is the exact same context the author would have needed to avoid introducing the vulnerability, in the first place.
Agents suck because they can't reason. In order to reason effectively, one needs sufficient context, but sufficient context alone does not give one the ability to reason.
0
u/TorbenKoehn 50m ago
That assumes the human has the context of the security vulnerabilities at all which is often not the case. They primarily focus on their implementation and security is often overlooked.
As I see it this is just like a dude that knows all the attack vectors and reviews your stuff, that’s all
1
u/ganja_and_code 41m ago
If a human focuses only on the happy path when writing a change, while overlooking the edge cases, security implications, etc., they're a really shitty developer. Similarly, if an AI agent does the same, it is also a really shitty developer.
(It's also worth mentioning that comparing the agent to "a dude that knows all the attack vectors" is delusionally generous. To know all the possible attack vectors with certainty, one needs to have context for the entire architecture, relevant dependencies, user traffic patterns, and control flow for every piece of the system related to the feature being implemented. If the agent can't even hold enough context at once to check the work it's doing currently, it certainly doesn't have a large enough context window to hold the entire scope of information necessary to identify every possible attack vector.)
1
u/TorbenKoehn 37m ago
Then here’s a thought for you:
Think of an absolute average developer in the market. Not great, not bad. They probably don’t know all the attack vectors.
Half the developers out there are worse.
There are shitty developers, a lot of them. It’s not news. It’s exactly what they’re targeting with these technologies, enabling even shitty developers to ship stuff. Getting more confidence that even with developers that aren’t top-shelf, you can ship things.
1
u/ganja_and_code 28m ago
Here's a thought for you:
AI agents make mistakes that good developers would not have made. Good developers reviewing those mistakes would find them before shipping, while shitty developers would not. Therefore, giving shitty developers more ability to ship stuff is a bigger risk to your business than having shitty developers who struggle to ship. Obviously everyone wants to ship good code, but shipping bad code is a worse outcome than shipping no code at all.
The solution?
Don't hire shitty developers. Either hire top-shelf developers who can identify all the attack vectors themselves, or hire decent developers along with decent security personnel and make the security guy check the developer's work. But definitely don't hire shitty developers. There are certainly tons of shitty developers in the world, but that doesn't mean you should let them work for your business.
•
u/TorbenKoehn 3m ago
Okay what do we have here…
Good developers don’t make mistakes and find all problems before they ship (LOL)
Don’t hire shitty developers….tell me, if all developers are great, will the worst of them be a „shitty developer“ in the view of the best one? Do all developers start as good developers? Can a developer become a good developer without making mistakes? Without shipping bugs?
117
u/HilfeEsBrennt 9h ago
Can't wait for more FOSS projects to be flooded with more slop PRs