The FFmpeg issue, known as "PixelSmash", allows for Remote Code Execution (RCE) when accessed remotely through a malicious video file, and Denial of Service (DoS) locally within applications such as Kodi, Emby, Nextcloud, PhotoPrism, OBS Studio, and others.
- This vulnerability was documented as CVE-2026-8461, and represents a heap out-of-bounds write in the MagicYUV decoder. This vulnerability was assigned a severity rating of 8.8 and can be taken advantage of by opening a malicious video file in either AVI, MKV, or MOV format.
- All applications that utilize FFmpeg's core library, libavcodec, for video processing/encoding are potentially impacted. However, exploitation for RCE will only occur if Address Space Layout Randomization (ASLR) is disabled or if an additional vulnerability is exploited to disable ASLR.
Cause and Impact:
According to researchers at software supply chain security firm JFrog, PixelSmash occurs due to how MagicYUV handles slices. Slices are individual sections of a video frame that can be processed independently from the rest of the video frame.
"PixelSmash causes a one-row heap buffer overflow in the MagicYUV decoder's slice handling. The buffer overflow is caused by an inconsistency between how the frame allocator computes chroma plane heights and how the MagicYUV decoder computes chroma plane heights," researchers from JFrog explain.
Users of various applications can be impacted by PixelSmash through several different means. Users can be impacted when viewing AVI, MKV, or MOV videos, when browsing a directory where these types of files exist (as thumbnails), or when running an automated media ingest process.
Multiple widely used media applications including Kodi, OBS Studio, PhotoPrism, GNOME/KDE/XFCE's thumbnail generator, and other tools utilizing FFmpeg with the MagicYUV decoder enabled are all vulnerable to PixelSmash. Slack, Discord, Telegram, and WhatsApp are also potentially impacted as they utilize FFmpeg to create server-side video previews; however, these have been excluded from testing.
Lead researcher at JFrog, Yuval Moravchik, demonstrated the ability to exploit PixelSmash in order to gain remote code execution access to both Jellyfin and Nextcloud instances (Nextcloud instances require movie preview to be enabled).
"To demonstrate the real-world implications, we successfully executed a full remote code execution against a Jellyfin v10.11.9 media server, the second most popular self-hosted media server (behind Plex), using Jellyfin's standard media library scanning pipeline," said JFrog.
"The actual attack path for Jellyfin is as follows: download of crafted MagicYUV AVI into media library, Jellyfin automatically invokes ffprobe for metadata extraction, out-of-bounds writes fire, avbuffer.free gets hijacked to system(), and the attacker's choice of commands execute as the Jellyfin service user."
Moravchik notes that the RCE exploit requires disabling of ASLR (Address Space Layout Randomization). He further states that CVE-2026-8461 itself cannot bypass this memory protection method. In theory, a separate information disclosure bug in FFmpeg's FlashSV decoder can be chained with PixelSmash in order to bypass ASLR.
Another possible attack vector is seeding a malicious torrent of a targeted video towards Jellyfin users who have configured their download settings to place downloaded content directly into Jellyfin's media library folder.
"Jellyfin monitors the file system in real time for new additions. When the malicious file appears, Jellyfin immediately launches an ffprobe metadata scan. During the scan the exploit fires. AVBuffer.free is hijacked to system(). The attacker's chosen reverse shell command executes as the Jellyfin service user."
While exploitation for RCE may be prevented or impossible, CVE-2026-8461 is still likely to provide reliable Denial of Service (DoS) conditions on affected systems.
As part of the research effort, the researchers identified Plex as a notable exception with regard to FFmpeg usage. Plex utilises a customised version of FFmpeg with all decoders disabled and a very limited whitelist applied, effectively blocking access to the MagicYUV decoder and therefore preventing exploitation of PixelSmash.
Following discovery, JFrog notified the FFmpeg security team on May 13th regarding the flaw. As a result, FFmpeg issued version 8.1.2 on June 17th, which resolves the defect. Additionally, Jellyfin updated their internal FFmpeg package and PhotoPrism is currently developing a blocklist for file formats intended to protect users from potential exploitation.
The Nextcloud development team was notified about the issue via HackerOne but did not choose to resolve it, as the flaw resides outside of Nextcloud.
Due to the widespread use of the MagicYUV decoder across hundreds of applications that "trust FFmpeg to securely handle untrusted input", PixelSmash presents a large attack surface, further characterising this as a supply chain type of vulnerability.
