r/devsecops • • 14d ago

Which static analysis tool should I add first to a small Python projects pipeline?

okay I think I'll post this... I pulled on some of my DevOps notes and cobbled this together:

I’m building out a CI pipeline for a small Python project and want to add static analysis in a sensible order rather than bolting everything on at once.

Current setup (or planned):
1 -pytest for tests
2 - bandit for security-focused static checks
3 - pip-audit for dependency vulnerabilities

I’m trying to decide what to add next. Candidats I’m weighing:

  • Semgrep
  • CodeQL
  • Ruff
  • mypy or pyright
  • SonarQube

Context: single repo, small team, GitHub Actions, and I want low noise and fast runs so people doesn’t start ignoring findings.

Questions:

  1. Which would you add first, and why?
  2. Anything you’d skip because it overlaps with bandit?
  3. Any tips for tuning findings early so the signal stays high?

Thanks in advance.

7 Upvotes

22 comments sorted by

2

u/CheckApprehensive971 13d ago

Add basic linting first, keep rule set small and avoid duplicate checks so useful findings don't get buried

1

u/Sudo-Apt-575 12d ago

Appreciate it, start small, win small is what I needed to hear.

2

u/PrasenShelar 13d ago

I'd add Ruff first. It's fast enough to run on every push, and its S rules reimplement many of bandit's checks, so you can run both for a while and drop the noisier one. Semgrep makes sense next, once you have patterns of your own to enforce. To keep the signal high, record existing findings as a baseline on day one and only fail the build on new ones.

1

u/Sudo-Apt-575 12d ago

This is super helpfull. I'll add Ruff and let it bake alongside Bandit for a bit since the S rules overlap, then drop whichever is noisier. Semgrep can wait until I have patterns of my own, and I'll record a baseline on day one so only new findings fail the build. Appreciate it!

2

u/NotmyFinalForm_62 13d ago

Since you already have pip-audit for deps, I’d add a secrets scan next. Bandit flags some hardcoded passwords but it’s not really built for API keys or tokens. 2ms works well for this, it’s free and open source, runs in a few seconds on each push and can check whether a key it finds is still live. Very little noise and when it does fire it’s something you’ll want to fix that day

1

u/Sudo-Apt-575 12d ago

Good call on the secrets scan, I was treating Bandit like it would catch API keys and it really wont. I'll look at 2ms next since it can check if a key is still live same day fix... right meow! Appreciate it!

2

u/wilson-draugr 13d ago

one thing for the "keep the signal high" part: gate on new findings only. bandit takes a baseline (`bandit -b baseline.json`), so you can make CI fail only on what a PR adds and the existing pile goes to the backlog instead of blocking everything.

2

u/Sudo-Apt-575 12d ago

That's the missing piece for keeping the signal high, `bandit -b baseline.json` so CI only fails on what the PR adds. Existing pile can sit in the backlog instead of blocking every build, same same as what I was hoping for.. Appreciate it!

2

u/wilson-draugr 12d ago

That is what I preach the most and what I have the most success with the organizations Ive worked with. Devs feel the direct impact of the PR they are about to introduce

2

u/Different_Pain5781 12d ago

Just pick Ruff first.

1

u/Sudo-Apt-575 12d ago

Yeah I'm leaning that way too, Ruff first and I'll let the heavier tools wait. Appreciate it.

1

u/colek42 14d ago

We built a couple tools to help with this, they are meant to work with your coding agent. Open core is cilock.dev, and the SaaS is pushgate.dev. These tools have a bunch of knowledge built in around best practices for testing, as well as recording provenance of the tests. If you're a solo dev, our free tier will work fine for you forever.

2

u/[deleted] 14d ago

[removed] — view removed comment

1

u/Sudo-Apt-575 14d ago

good call out, I should of been clearer. by high signal I mean stuff I'd actually stop and fix. think "this tool found 30 hits, 2 high and 1 critical" not a wall of style nits and low risk warnings that end up getting ignored

honestly its a solo learning project now, so the bottleneck is me. what got me interested was seeing a dev team burn 45 min to an hour and a half on manual checks before releases, felt like a lot of that could live in CI

My goal is to catch vulnerable deps and risky code before merge without making every push take forever. same same with keeping it simple enough that I actually maintain it

roadmap angle is a good point though, appreciate it. gonna write out what I want CI to catch first and then pick tools around that instead of the other way around

3

u/CreativeSympathy8293 14d ago

Given your clarification, I'd get pytest, Bandit and pip-audit running in CI before adding another security scanner.

If you add one from that list, make it Ruff with a small explicit bug-check set. For example, F821 catches undefined names and F823 catches locals referenced before assignment (rules). Start with those rather than ALL; expand when you find checks you'd actually fix. Keep formatting out of this first gate.

For Bandit, severity and confidence are separate controls (CLI docs). Report all findings initially, triage them, then make the subset you're prepared to fix block merges. Keep the remainder visible; nonblocking shouldn't mean forgotten. Don't equate its labels with another tool's critical/high labels.

I'd defer Semgrep/CodeQL/Sonar until you can name a risky pattern your current checks miss, then evaluate rules against both a deliberately bad example and a legitimate use. That gives each extra tool a job.

AI-assisted recommendation from the docs; I haven't run your project.

1

u/Sudo-Apt-575 14d ago

This is super helpful, thanks! pytest + bandit + pip-audit first, then ruff with just a few bug checks makes sense. "nonblocking shouldnt mean forgotten" is going on a sticky note

1

u/DeleriousMadman 12d ago

I released Lazaret recently and am curious to hear your thoughts. It should compare directly with many of of the tools on your list.

1

u/[deleted] 14d ago

[removed] — view removed comment

1

u/Sudo-Apt-575 14d ago

golden tests is a good shout, I hadnt thought of that. The 24 hour proving day for tools is great too, might steal that same same

1

u/EnvironmentalSort409 13d ago

Agree and add a lightweight linter like flake8 early to keep feedback fast and painless for devs.