How to Reduce Security Review Time by Connecting LLM APIs to Open Source Vulnerability Scanning

Open source is convenient, but it is equally risky. According to a 2025 survey, as AI began writing code on behalf of humans, the bug occurrence rate soared by 41% compared to the previous year. For security professionals who must personally review tens of thousands of lines of external libraries, this is nothing short of a disaster. Since it is impossible to read every line of code, we must turn AI into our ally. Here is a summary of how to build a smart security workflow that operates like Project Glasswing.

Installing an AI Security Engine in GitHub Actions

Automating security reviews can eliminate over 10 hours of simple, repetitive tasks each week. It also prevents mistakes that humans might overlook during manual scanning. Build a pipeline in the GitHub Actions environment that calls an LLM API to perform real-time scans every time a Pull Request is submitted. The key strategy is to separate identification from auditing rather than simply asking questions.

• Protecting API Keys: Register your LLM_API_KEY in GitHub Secrets. Storing it in a Libsodium encrypted repository prevents accidents where the key might be leaked externally.
• Setting Target Directories: You don't need to look at every file. Use a path-filter in your YAML configuration to select and scan only sensitive directories where a breach would be catastrophic, such as src/auth or lib/core.
• Cross-Verification: In the first stage, have the LLM analyze the code structure and write notes; in the second stage, run repeated scans at least three times based on those notes. Since AI can occasionally produce "hallucinations," it is necessary to cross-check results multiple times.

Once this setup is complete, security managers only need to check the security reports summarized by the AI instead of wading through tens of thousands of lines of code.

Filtering the Real Threats with CVSS and EPSS

While AI tools are great at finding vulnerabilities, they also produce many false positives. If 100 issues are found and 15 of them are fake, the development team will inevitably become frustrated. To avoid wasting limited development resources, you need criteria to filter out real threats. Prioritize by combining CVSS 4.0 scores with EPSS metrics, which indicate whether an exploit is currently occurring in the wild.

1. Check the Base Score: Look at the technical severity first using the CVSS 4.0 Base Score.
2. Adjust the Environmental Score: Subtract or add points depending on whether the code is an external internet-facing feature or a function running only on an internal network.
3. Cross-Reference Exploit Probability: Query the EPSS database to check if the vulnerability is currently popular among attackers. If the exploit probability exceeds 50%, you should drop everything else and patch it immediately.

Focusing only on "Urgent" ratings of 9.0 or higher will significantly raise your security level. Reducing unnecessary fix requests also naturally reduces friction with the development team.

Validating and Deploying Patch Code in a Sandbox

Fixes suggested by AI may look perfect on the surface, but they sometimes break perfectly fine features. Companies like Shopify use AI, but they do not trust generated code blindly. You must have an automated procedure to verify if patch code is safe within isolated environments like Firecracker or gVisor.

• Create an Isolated Environment: Use the sbx CLI to spin up a MicroVM with the exact same runtime environment as your current service.
• Attack Simulation: Verify if the vulnerability is actually exploitable with an attack script, then apply the AI-provided patch and attack again to check if it is blocked.
• Functional Testing: Run existing unit tests with the patch applied. If you've blocked the security hole but users can't log in, the fix is meaningless.

These safeguards prevent accidents where code that is "mostly right but subtly wrong" ends up on production servers.

Writing Reports That Open Source Maintainers Accept Immediately

Don't stop at just fixing your own service. It is also the security manager's responsibility to report flaws in the open source itself to the upstream project. Maintainers are busy people, so you must provide clear evidence. Use GitHub's PVR channel to deliver reports responsibly.

Clearly state the vulnerability type and location in the title. Attaching a reproduction path that anyone can follow, along with screenshots, is fundamental. The best approach is to include the fix code you previously verified in the sandbox. Reducing the review time for maintainers drastically increases the probability that your patch will be merged. A single, well-crafted report proves a company's technical prowess and can even lead to securing an official CVE number.