Veridical.dev
Veridical15 min readArticle

Secrets in Code: Why Developers Need More Than Scanners

← All articles

Secrets in Code: Why Developers Need More Than Scanners

Isometric illustration of contextual secret detection

Secrets in code are credentials and keys accidentally stored as literal values in a codebase: API keys, passwords, tokens, and private keys that end up where attackers can find them. The immediate fix is twofold: stop secrets from reaching your repository through push protection and managed secret stores, and rotate anything already exposed. GitHub’s push protection and CISA’s rotation guidance are the two reference points worth building a workflow around.

TL;DR:

  • A 2025 analysis found exposed credentials in 65% of GitHub repositories tied to Forbes AI 50 companies, including access to private models and data.
  • CISA recommends rotating API keys at least every 90 days; use least privilege and short lived OIDC tokens where supported to limit exposure.
  • Scanning should cover commit history, CI logs, build artifacts, package registries, and collaboration tools, because removing a credential from current code leaves older commits intact.
  • Entropy and pattern matching miss unfamiliar credentials and flag harmless strings; prioritize context aware detection and live credential checks, then tune allowlists for fixtures.
Veridical
Catch Risky Changes Before Merge
Veridical reviews GitHub pull requests with verified findings and inline evidence to help teams spot defects that scanners may miss.
Explore evidence-based PR reviews

Table of Contents

1. What counts as a secret in your codebase

A secret is any value that grants access and was never meant to be readable by someone outside your trust boundary. The category is broader than most developers assume when they first set up scanning.

  • Passwords and database credentials hardcoded in connection strings or config files.
  • API keys and tokens for third-party services, often pasted directly into a script during debugging.
  • Private keys and certificates, including SSH keys and TLS certificates committed alongside public assets.
  • Encoded or derived secrets, such as base64 blobs or values split across environment templates.

Secrets show up as plain string literals, inside .env files that got committed by mistake, in Infrastructure as Code templates with default values left in, or buried in sample config files that were copied from a real one. Encoded values deserve particular attention: a base64 or hex blob might just be a derived secret, which is why GitHub’s guidance on Actions recommends registering derived secrets for redaction rather than assuming only the original value needs masking.

2. Why finding secrets in source code is genuinely hard

Detection tooling has to walk a narrow line between catching real exposures and drowning developers in noise. Entropy-based heuristics, which flag strings that look sufficiently random, generate a steady stream of false positives because plenty of legitimate strings (hashes, UUIDs, test fixtures) also look random. The reverse problem is just as damaging: most secret formats carry no distinguishing prefix or pattern, so a scanner tuned only to recognize, say, a specific cloud provider’s key format will miss everything else.

Context helps close that gap. Research on this exact problem found that context-aware methods, which weigh how a string is used in surrounding code rather than judging the string in isolation, substantially outperform simple rule-based or entropy-only approaches, though their advantage can shrink when applied to data the model has not seen before, according to research published in Empirical Software Engineering. That gap between benchmark performance and real-world generalization is the reason vendor claims about detection accuracy deserve scrutiny before they inform a team’s tolerance for blocking behavior.

The stakes are not theoretical. A 2025 analysis found that 65% of GitHub repositories tied to Forbes AI 50 companies had exposed secrets, granting access to private models and data at organizations collectively representing more than $400 billion in value. Detection limits at that scale translate directly into breach risk.

2. Why finding secrets in source code is genuinely hard — overview diagram

3. How secret scanning actually works

Secret scanning is not a single check. It is a pipeline with distinct stages, and knowing which stage caught a finding tells you how much to trust it.

  1. Entry points: scans run at pre-commit (local, before code leaves your machine), at push time (blocking before it reaches the remote), on pull requests (reviewing the diff), across commit history (catching what already landed), and on build artifacts (binaries, container images, package registry uploads).
  2. Detection families: regex and rule-based matching is fast and cheap but brittle against new formats; entropy filters catch unstructured random-looking strings but produce heavy false-positive rates; machine learning classifiers trained on context, similar to CodeBERT-style models, improve precision by weighing surrounding code but require ongoing retraining; LLM-based prompting can reason about intent but tends toward caution, flagging more candidates than it should.
  3. Validation: automated metadata checks confirm a candidate matches an expected format, deterministic verification calls the relevant API to check if a credential is still live, and manual triage handles anything ambiguous that survives both.
  4. Reporting and remediation: findings route to alerts, PR comments, or ticketing systems, and the strongest setups wire automated rotation hooks directly into the alert so a confirmed live secret gets revoked without waiting on a human to act first.

GitHub documents push protection as a preventive control that blocks pushes containing potential secrets before they ever enter repository history, which removes the far more expensive job of scrubbing git history after the fact.

Pro Tip: Treat a scanner’s first few weeks of output as a tuning exercise, not an incident queue: most of what it flags on day one will be tested fixtures and placeholders.

4. Where secrets actually leak across your development lifecycle

Source files get the most attention, but they are one surface among several. Commit history is a persistent risk because a secret removed from the latest commit still lives in every earlier one unless history itself is rewritten. Infrastructure as code templates, including Terraform and CloudFormation files, often ship with default credentials or example values that get copied into production configs without modification.

  • Repositories and commit history, including accidental pushes to feature branches that never get cleaned up.
  • CI/CD pipelines and run logs, where GitHub Actions output or third-party action logs can print a secret in plain text during debugging.
  • Build artifacts and package registries, where a credential baked into a binary or container image ships far beyond the original repository.
  • Collaboration tools and tickets, where a developer pastes a connection string into a support ticket or chat thread while troubleshooting.

Industry analysis of secret sprawl confirms this spread: secrets commonly appear outside source code entirely, turning up in collaboration platforms, logs, and build artifacts throughout the development lifecycle, not just in the repository a team thinks to scan first.

5. Prevention and remediation steps that actually hold up

Most of this comes down to making the secure path the default path, not an extra step developers have to remember.

  1. Enable push protection and pre-receive checks at the organization level so a secret never reaches the remote repository in the first place, and add pre-commit hooks so developers get the same feedback locally, before a push attempt even happens.
  2. Store secrets in a managed secret store rather than environment files, and prefer short-lived OIDC tokens over long-lived keys wherever your CI provider supports them, since a token that expires in minutes limits the damage of an exposure dramatically more than a key that works indefinitely.
  3. Rotate credentials on a schedule: CISA recommends rotating API keys at least every 90 days, paired with least-privilege scoping so a compromised key grants the smallest possible blast radius.
  4. In CI specifically, register every secret a workflow actually uses so it gets masked in logs, avoid any step that could print a secret value to console output, and audit third-party actions before granting them access to repository secrets.
  5. When a secret does leak, the sequence matters: remove the literal value from the current code, rotate or revoke the credential at the source (not just in your repo), search full commit history for other occurrences, delete any CI logs that captured the value in plain text, and notify whoever owns the affected system.

Pro Tip: Revocation comes before cleanup, every time: a secret sitting in old commit history is only dangerous if it still works, so rotating it first buys you time to clean up the rest without racing an attacker.

6. What to expect when detection tooling gets it wrong

No detection approach eliminates both false positives and false negatives at once, and planning around that trade-off saves teams from abandoning scanning after a noisy first month. Test fixtures, documentation examples, and placeholder values (your_api_key_here) account for a large share of false positives, and most mature scanners let you allowlist specific patterns or paths to filter them out.

  • False negatives often come from encoded or transformed secrets, or from credentials embedded inside compiled binaries rather than source text.
  • Benchmark inflation is a real risk: datasets with duplicate records or overlap between training and test data can make a detector look far more accurate than it performs on genuinely unseen code, a concern researchers have flagged directly about datasets in this space.
  • Blocking strictness is a judgment call: push protection that blocks too aggressively trains developers to work around it, while protection that is too permissive defeats its purpose.

7. Where evidence-based PR review fits alongside secret scanning

Secret scanning and push protection catch a specific, narrow category of risk: literal credential values. They do not evaluate whether the surrounding code change introduces a logic flaw, a performance regression, or an unsafe dependency on that credential once it is stored correctly. That is the gap evidence-based pull request review is built to close.

  • Risky diffs can be flagged with inline evidence tied to the specific lines responsible, so a reviewer can see why a change is flagged, not just that it was.
  • A calibrated advisory score, projected from real-defect F1 metrics, can give teams a triage signal that cuts down manual review time on pull requests that pose little risk.
  • Push protection stops a secret from being committed; secret scanning finds one that already made it through; our evidence-based PR reviews look at the broader change, including how a newly introduced credential gets used, to catch risky merges that scanning alone would pass.

Secret scanning tools generally fall into three tiers of effectiveness, and the differences line up closely with the detection families covered earlier. Pure regex and entropy-based scanners are the fastest to deploy and catch well-known key formats reliably, but they struggle with anything that does not match a predefined pattern, which is most secrets. Platform-native scanning, such as GitHub’s own secret scanning and push protection, benefits from direct integration at push time and partnerships with credential providers that let GitHub verify whether a flagged string is a live, active secret rather than a dead one, which meaningfully cuts false positives for supported providers.

Third-party and standalone scanners add value mainly through broader pattern libraries and historical scanning across entire git histories, useful for organizations onboarding a repository that predates any scanning policy. The newest category, context-aware and ML-assisted detectors, shows the strongest reported precision gains in academic evaluation, but as covered above, that performance can soften outside the benchmark it was trained on. The practical implication for choosing a tool is to weigh its validation stage as heavily as its detection stage: a scanner that confirms a candidate is a live, working credential before alerting will generate far less noise than one that only pattern-matches, regardless of which detection family it uses underneath.

9. Integrating secret detection beyond GitHub Actions

GitHub Actions gets the most documentation, but secret detection needs to run wherever code moves, and most CI providers support the same core integration points. GitLab CI, Jenkins, CircleCI, and Azure Pipelines all support pre-commit hook integration, pipeline-stage scanning jobs, and artifact scanning before a build gets published to a registry. The pattern that matters most is running a scan as early as possible in the pipeline and failing the build on a confirmed, high-confidence finding rather than waiting until deployment.

Secret scanning stages from commit to artifact

For teams running mixed infrastructure, a few integration habits carry across every platform: register every secret a pipeline consumes so the platform can mask it automatically in logs, scope service-account credentials narrowly per pipeline rather than sharing one broad credential across jobs, and audit any third-party plugin or action with access to secrets the same way you would audit a new dependency. Partner guidance on AI-assisted development teams echoes this: as more of the pipeline becomes agentic or AI-generated, governance around credential handling and code guardrails needs to scale alongside the automation itself, not trail behind it.

10. Why entropy alone cannot reliably spot a secret

Shannon entropy measures how unpredictable a string looks, and it was an obvious starting point for secret detection because real credentials tend to be high-entropy by design. The problem is that plenty of legitimate strings share that property: hashes, UUIDs, session identifiers, and even some natural-language strings in certain encodings can trigger the same threshold a genuine API key would.

Industry analysis of secrets sprawl is direct about this limitation: entropy alone is an unreliable signal, and combining it with context, such as how a variable is named, how it is used in surrounding code, and whether it is assigned to something that looks like an API call, dramatically improves the signal-to-noise ratio for flagged candidates. Pattern recognition faces a parallel challenge: most secret formats carry no identifying structure at all, so a scanner built around known prefixes for a handful of major cloud providers will miss custom tokens, internal credentials, and anything issued by a service that does not follow a public convention. The practical takeaway for any team evaluating a scanner is to ask specifically how it handles secrets with no recognizable format, since that is where most tools quietly lose coverage.

Rolling out scanning without stalling your team

Start by locking down your main branches with push protection before touching anything else. Keep rules high-signal, automate remediation for clear-cut matches, and route ambiguous flags to a human rather than blocking on every uncertain case. Strictness that outpaces trust just gets routed around.

— Łukasz

Reduce risky merges with evidence-based PR review

Secret scanning and push protection catch exposed credentials. They were never built to tell you whether the pull request introducing that credential is otherwise safe to merge. That is where our evidence-based PR review comes in: verified findings with inline evidence, a calibrated advisory score built on real-defect F1 projections, and a read on repository context so a review accounts for callers and dependencies, not just the diff.

Veridical

Open-source maintainers can start with our free tier, and teams ready for paid review capacity can compare the Standard plan at $39 per month and Pro at $69 per month.

FAQ

What are secrets in programming?

Secrets in programming are credentials and keys, such as passwords, API keys, tokens, and private keys, that grant access to a system and need to stay confidential. They become a risk specifically when stored as literal values inside source code or configuration files instead of a managed secret store.

What are examples of secrets?

Common examples include database passwords, cloud provider API keys, OAuth tokens, SSH private keys, and TLS certificates. Encoded or derived values, like a base64-encoded credential blob, also count and often get missed by scanners looking only for the original format.

How can I use GitHub secrets in my code?

GitHub Actions lets you store values as encrypted repository or organization secrets and reference them in workflow files without exposing the raw value in your code. GitHub’s guidance recommends masking sensitive data, registering derived secrets for redaction, and preferring short-lived OIDC tokens over long-lived stored credentials where your provider supports them.

What is a secret scan?

A secret scan is an automated check that looks through source code, commit history, or build artifacts for credentials that should not be exposed. GitHub’s push protection is one form of this, blocking a push at the moment it detects a potential secret rather than finding it after the fact.

Sources