# Integer overflow in CipherUpdate

Integer overflow in OpenSSL EVP_CipherUpdate with INT_MAX input length yields negative outlen, causing out-of-bounds memory access in callers.

- Payout: hidden
- Severity: high
- Program: Internet Bug Bounty
- Found by (vendor credit): Paul Kehrer
- Found by: reaperhulk
- Disclosed: 2021-04-08
- CVE: CVE-2021-23840, CVE-2020-36242
- Write-up quality: 6/10
- Original: https://hackerone.com/reports/1113025
- Library page: https://bugbountylibrary.com/r/h1%3A1113025
- Tags: area:oss domain:oss impact:dos kind:bounty platform:hackerone sev:high technique:source-review vuln:memory-corruption

**Target:** OpenSSL EVP_CipherUpdate (AES-CBC) API
**Root cause:** EVP_CipherUpdate adds block padding to the input length in a signed int, so passing INT_MAX overflows to a negative output length instead of erroring.
**Steps:**
1. Allocate inbuf of INT_MAX (2147483647) bytes and outbuf of 2147483648 bytes.
2. EVP_CipherInit_ex with EVP_aes_128_cbc, zero key/iv, encrypt=1.
3. Call EVP_CipherUpdate(ctx, outbuf, &outlen, data, 1) to prime the context.
4. Call EVP_CipherUpdate(ctx, outbuf, &outlen, inbuf, INT_MAX).
5. Observe outlen is negative and res==1 (success) instead of an error.
**How found:** Researcher reasoned about signed int arithmetic in the cipher update path when input length approaches INT_MAX, then wrote a C reproducer to confirm the negative outlen.
**Insight:** When auditing crypto APIs, test boundary lengths near INT_MAX/INT_MIN on functions that add padding or block sizes to a caller-supplied int length; a negative outlen returned with success status is the tell.