Blog Posts
LLM Sandboxing: Early Lessons Learned
By Matt Hamilton
March 20, 2023
About two weeks ago, we launched our research project and text-based AI (sandbox) escape game Doublespeak.chat. We give the OpenAI’s Large Language Model (LLM, A.K.A. ChatGPT) a secret to keep: its name. The player’s goal is to extract that secret name. We believe we'll never win the cat-and-mouse game, but we can all have fun trying!
Jailbreaking LLM (ChatGPT) Sandboxes Using Linguistic Hacks
By Matt Hamilton
March 8, 2023
We had a simple idea: we prime an LLM (Large Language Model), in this case ChatGPT, with a secret and a scenario in a pre-prompt hidden from the player. The player's goal is to discover the secret either by playing along or by hacking the conversation to guide the LLM's behavior outside the anticipated parameters.
GraphQL Security Testing Without a Schema
By Alex Leahu
August 19, 2022
One of the main obstacles of a black box GraphQL security review is getting good coverage of the exposed functionality. Anyone who has reviewed a GraphQL API will have seen many requests that look something like this:
This isn't fun to look at, but more importantly, getting the coverage you need isn't feasible. You would need to spend a lot of time reviewing each request to determine queries, arguments, and fields. It also doesn't make much sense to test GraphQL endpoints by manipulating raw HTTP requests, and it's much more suitable to use tools like GraphiQL. This is only an issue if introspection is disabled. Otherwise, you could point GraphiQL (or similar tools) to the GraphQL endpoint and have a fully populated schema to aid the construction of queries.
I wondered if it would be possible to passively observe traffic and piece together a GraphQL schema based on the queries that went through Burp Suite. If I could do this, it would let anyone interact with a GraphQL API through GraphiQL even without having the schema. Having a GraphQL schema is a considerable improvement over working with raw HTTP requests. After some trial and error, I developed a practical approach to do just that.
Make Search Engines Great Again!
By Alex Leahu
July 8, 2022
Search engines are part of everyday life and help us quickly find information we are looking for. But anyone who has used search engines for a long time realizes its pitfalls. Companies have figured out how to optimize their websites and fight for the top spaces of your search results. As a consequence, we end up with results that are not necessarily high quality and instead contain websites that have gamed the system to get on top.
In our field (information security) we use search engines on a daily basis to look up technical write ups, documentation, vulnerability details, and more. Wouldn’t it be great if we could at least improve our tiny corner of the internet to become more productive and surface high quality information that is actually relevant?
$ email blame
By Matt Hamilton
June 20, 2022
A recent Hacker News post made me realize that something I've been doing for many years isn't as uncommon as I had thought.
I have a catch-all email address for one of my domains. *@mydomain.example all goes to one folder.
Stop Storing Secrets In Environment Variables!
By Matt Hamilton
January 28, 2022
Did that get your attention? Good. The goal of this short post is to evangelize engineering teams to end the psychosis that is storing secrets in environment variables.
A Primer for Testing the Security of GraphQL APIs
By Alex Leahu
October 22, 2021
Whether you're a penetration tester, security engineer, or bug bounty hunter, it can be incredibly helpful to know how to find vulnerabilities in a GraphQL API. This post will introduce you to GraphQL and its functionality from the perspective of someone performing a security assessment.
The post will not focus on how to securely implement a GraphQL API, although you can extrapolate details that’ll help you in doing so. Additionally, although I will draw parallels to familiar topics like REST and SQL, other concepts may be new.