# Machine Learning Attack Series: Backdooring Keras Models and How to Detect It

A Keras .h5 model can be backdoored with a Lambda layer that executes arbitrary code (downloads malware, leaks hostname) on load, and ModelScan detects it.

- Payout: Research
- Severity: medium
- Program: Keras
- Write-up by: Johann Rehberger
- Disclosed: 2024-05-18
- Write-up quality: 6/10
- Original: https://embracethered.com/blog/posts/2024/machine-learning-attack-series-keras-backdoor-model/
- Library page: https://bugbountylibrary.com/r/blog%3Aembracethered-com-blog-posts-2024-machine-learning-attack-series-keras-backdoor-model
- Tags: area:ai domain:ai impact:data-leak impact:rce kind:research platform:research sev:medium tech:python technique:source-review vuln:dependency-confusion vuln:deserialization vuln:rce

**Target:** Keras .h5 model file (legacy HDF5 format)
**Root cause:** Legacy Keras .h5 model format deserializes Lambda layers containing arbitrary Python callables, and safe_mode=True is not enforced for older formats, so loading the model executes attacker code.
**Steps:**
1. Load original model: `keras.models.load_model("huskymodel.h5")`
2. Define malicious function `delicious_layer(x)` that prints, opens HTTPSConnection to attacker host, sends `GET /l.png?lambdahost={HOSTNAME}`, writes response to disk
3. Wrap in `keras.layers.Lambda(delicious_layer)` and `temp_model.add(lambda_layer)`
4. Compile with original optimizer/loss/metrics and `temp_model.save("huskymodel-lambda-backdoor")` (legacy format, not .keras)
5. Victim runs `keras.models.load_model("huskymodel-lambda-backdoor")` — code executes immediately on load
6. Detect with `modelscan -p huskymodel-lambda-backdoor` which flags unsafe Lambda operator in keras_metadata.pb
**How found:** Author reasoned from prior pickle deserialization research and Microsoft's 'Abusing ML model file formats' post that Keras Lambda layers are serialized callables; tested by adding a Lambda layer and observing code execution on load, then noted .keras v3 safe_mode would block it but legacy formats don't.
**Exploitation:** Attacker with write access to a shared model file (network share, HF repo) adds a Lambda layer and re-saves in legacy .h5 format; any victim loading the model executes the payload, enabling malware download and hostname exfiltration via the outbound request.
**Insight:** Keras safe_mode only applies to the v3 .keras format; legacy .h5/.hdf5 models still execute Lambda layers on load, so check model file extensions and scan for Lambda/custom layers in keras_metadata.pb before loading untrusted models.