Getting a talk accepted at Black Hat is a defining moment for any security company, and we're thrilled to share the key insights from our presentation at Black Hat USA 2025 in Las Vegas. Our 40-minute briefing detailed how we detected and responded to one of the most significant CI/CD supply chain breaches of 2025.
The Friday Afternoon That Changed Everything
Our presentation began with a story that resonated with every security professional in the room. It was Friday afternoon, March 14th, when we were wrapping up our last meeting of the week. Then came the Slack notification that would consume our weekend – and ultimately expose a supply chain attack affecting thousands of organizations worldwide.
The tj-actions/changed-files GitHub Action, a seemingly innocuous tool used to detect file changes in pull requests, had been weaponized. This wasn't just any component – it was actively used in over 23,000 public repositories, including those belonging to GitHub, Hugging Face, HashiCorp, Meta, and Microsoft.
The Attack: Hiding in Plain Sight
The sophistication of this attack was remarkable. The attackers didn't just compromise the action – they employed a technique called "imposter commits." These are commits that don't actually exist in the target repository but instead live in a fork. Through GitHub's API architecture, release tags can point to these ghost commits, making them accessible while remaining invisible in the repository's commit history.
During our presentation, we demonstrated the attack chain:
- Attackers created a fork of the legitimate repository
- Added malicious code to their fork
- Updated all release tags (v35, v44, etc.) to point to their malicious commit
- Every workflow using these tags automatically executed the backdoor code
The malicious code itself was elegantly simple yet devastating. It downloaded a Python script called memdump.py from a public GitHub Gist, which then dumped the memory of the Runner.Worker process – the component that holds all CI/CD secrets during workflow execution. These secrets were then exfiltrated directly in the build logs using double base64 encoding to bypass GitHub's automatic secret masking.
Detection Through Baseline Monitoring
What made this presentation particularly compelling was our live demonstration of how the attack was detected. StepSecurity Harden Runner had established a baseline by monitoring thousands of workflow runs. When the compromised action suddenly made an outbound call to gist.githubusercontent.com – a destination never seen before in those runs – it triggered an immediate alert.
The audience was particularly engaged when we showed the actual detection screenshot, displaying how a single anomalous network connection exposed an attack that traditional security tools had missed. The attackers were clever, using GitHub-owned domains with high reputation scores that most EDR solutions would trust implicitly.
The Domino Effect: A Chain of Compromises
Perhaps the most alarming revelation was the chain of compromises that led to this breach. Our investigation, detailed in the presentation, revealed a complex attack chain:
- The tj-actions/changed-files compromise originated from a compromised reviewdog/action-setup action
- The reviewdog action was compromised using a stolen Personal Access Token (PAT) belonging to a reviewdog maintainer
- That maintainer's PAT was stolen from the spotbugs/spotbugs repository where they had access
- The spotbugs/spotbugs repository itself was compromised using another PAT that was stolen from the spotbugs/sonar-findbugs repository
- The initial breach point was a Pwn Request vulnerability in the spotbugs/sonar-findbugs repository
This domino effect illustrated a critical vulnerability in the GitHub Actions ecosystem: the interconnected nature of actions and the broad permissions often granted to maintainers created a perfect storm for supply chain attacks. The reviewdog organization had a policy where anyone who submitted a merged pull request automatically received write access to the organization's repositories. When one of these maintainers had their PAT compromised through the spotbugs chain, it provided the attackers with the keys to compromise the entire reviewdog organization, which then led to the tj-actions breach.
Live Demonstration: Seeing is Believing
The highlight of our presentation was a live demonstration where we recreated the entire attack. We showed the audience two scenarios:
Normal Scenario
A typical workflow run with clean build logs and stable network baseline.
Compromise Scenario
The same workflow after tag manipulation, showing:
- The malicious imposter commit execution
- Real-time credential theft
- Secrets appearing in build logs in double base64 encoded format
- Network baseline becoming unstable due to the anomalous connection
The leaked AWS access keys and GitHub tokens in the build logs drove home the severity of the threat.
Industry Impact and Response
The presentation highlighted how this moved from an open-source hygiene issue to a matter of national security importance when CISA published a cybersecurity advisory about the compromise. For nearly 24 hours, the compromised action remained available, actively leaking credentials from thousands of organizations.
We shared the timeline of our response:
- 5:00 PM UTC: Initial investigation confirmed the compromise
- 8:00 PM UTC: Community notification and identification of affected repositories
- Next day, 2:00 PM UTC: GitHub removed the action entirely
- Next day, 10:00 PM UTC: GitHub restored the cleaned repository
Lessons Learned and Recommendations
Our presentation concluded with four concrete recommendations that resonated strongly with the audience:
- Implement Security Monitoring for CI/CD Runners: Organizations have monitoring for laptops and production environments but often have zero visibility into CI/CD runners.
- Enforce an Action Allowlist: With over 25,000 actions in the GitHub Marketplace, curation is essential.
- Pin Actions to Specific Commit SHAs: Organizations using commit SHAs instead of mutable tags were not affected by this incident.
- Develop an Incident Response Plan: We showed real examples of the challenges organizations faced in identifying compromised workflows and rotating credentials.
The Attackers' Sophistication
What particularly captivated the Black Hat audience was our analysis of the attackers' evasion techniques:
- Using legitimate GitHub domains to avoid detection
- Embedding exploit code to avoid network connections
- Employing imposter commits to keep repository history clean
- Impersonating legitimate users like renovate-bot
As we noted in our presentation: "None of the traditional security tools would flag any of this as malicious, and that's the terrifying brilliance of this attack."
Looking Forward
The presentation ended with a sobering thought that resonated throughout the conference: "These attack blueprints are now public. The next one won't just dump secrets in build logs. The next one won't just compromise two actions, and it might already be in your CI/CD pipeline."
As supply chain attacks become increasingly sophisticated, the security community must evolve beyond signature-based detection to behavioral analysis and anomaly detection.
For those interested in diving deeper into the technical details, you can download our complete Black Hat 2025 presentation slides here.
This Black Hat presentation represents a milestone for StepSecurity and reinforces our commitment to securing the software supply chain. The validation from the security community and the engaged discussions that followed our talk confirm that CI/CD security is no longer a nice-to-have but a critical component of any organization's security posture.
