The Moat for AI
Originally posted here. These are my personal opinions. If you want to add to the discussion, we welcome PRs to this website here.
Originally posted here. These are my personal opinions. If you want to add to the discussion, we welcome PRs to this website here.
DeepSeek R1 has arrived and reinforced the fact that no company has a moat on Generative AI. The famous leaked Google memo was accurate with respect to the performance. However, Google does have a security moat. They have 20 years of experience managing AI risk with spam & other models. That’s not easily reproducible. These skills in maintaining models in adversarial environments mostly translates to LLMs. The technical gap between spam and Generative AI allows for safety teams at Anthropic, OpenAI, and other younger companies to compete in the LLM market with trusted AI systems using new ideas. The most valuable part of the AI industry is the security teams.
AI constantly does things that it’s not supposed to do. A banking chatbot for Chase shouldn’t discuss the weather in Mongolia, nor should it insult its customers. An “agent” that emails company secrets when fed a doctored image is worse than nothing. As social media has found, you do more business when people want to be on your platform. Your AI will do more business when it’s not accidentally selling trucks for $1.
To make Generative AI valuable we need guardrails. The moat is the people who manage the AI to keep it on the straight and narrow. This is extremely difficult. The value of AI companies like Anthropic, Google, OpenAI, Meta, and Microsoft are their trust and safety teams. They act as the primary defense securing your model against prompt injections and other threats. You pay for a managed, effective security layer integrated with a top of the line LLM.
The other option is to do the security layer yourself. Deploying an open source model from a small startup like Mixtral or DeepSeek is opting to do all the security yourself. This might be needed if you require a particular fine tuning. The savings of having a small model efficiently deployed could disappear when you add the costs of making sure it does what it is supposed to and nothing else. Of course what is likely the best option is to pay one of the guardrail companies, like HiddenLayer to do most of it for you.
Either with a large LLM vendor or with a custom built solution, the final security layer will be built and managed by your team in house. Someone at that GM dealership should be monitoring their AI, even if they use the latest and greatest from Anthropic. Generic LLMs can be instructed to stay within bounds, but the end customer defines the bounds. As we know from years of AI security, even if we carefully write the objective function for an AI, a simple mistake could ruin it.
To bring it to a traditional security context, think of authentication. In this analogy, the service of an AI security team at Google or HiddenLayer is akin to using a trusted third party cryptography library. You can opt to “roll your own crypto”, which is needed in some cases, but only experts should go here. Either way, you will however need security practices internally to manage the passwords, cookies, or tokens yourself.
While the market may be panicking now, it will come to its senses soon. AI security and the people who manage it is the moat. A company that has agile & secure systems that do what they’re told to do at a reasonable price will beat DeepSeek R1.
Originally posted here. These are my personal opinions. If you want to add to the discussion, we welcome PRs to this website here.
Generative Red Team History
Threat Modeling LLM Applications Before we get started: Hi! My name is GTKlondike, and these are my opinions as a cybersecurity consultant. While experts fr...
Largest annual hacker convention to host thousands to find bugs in large language models built by Anthropic, Google, Hugging Face, NVIDIA, OpenAI, and Stabil...
The Spherical Cow of Machine Learning Security
Prompt Detective Announcement
Disclaimer: This does not reflect the AIV as a whole, these are my opinions and this was my response.
AI and ML is already being used to identify job candidates, screen resumes, assess worker productivity and even help tag candidates for firing. Can the inter...
The Red Team Village and the AI Village will host a panel from different industry experts to discuss the use of artificial intelligence and machine learning ...
Automate Detection with Machine Learning
A few useful things to know about AI Red Teams
Automate Detection with Machine Learning
Generative Art at AI Village DEF CON 30
Welcome to the second post in the AI Village’s adversarial machine learning series. This one will cover the greedy fast methods that are most commonly used. ...
Originally posted on Medium - follow @sarajayneterp and like her article there
Welcome to AI Village’s series on adversarial examples. This will focus on image classification attacks as they are simpler to work with and this series is m...