Target OS: Windows / Linux (a CUDA driver is required)
What Is NVENC
NVENC is the video encoder built into NVIDIA GPUs. It is much faster than encoding on the CPU and uses very little CPU. In exchange, it tends to compress a little less well than a CPU encoder.
| Item | NVENC Hardware | libx264 Software |
|---|---|---|
| Speed | About 2–3× faster (measured at 1080p; see below) | Baseline |
| CPU usage | Low (GPU processing) | High |
| Compression efficiency | h264_nvenc: 2–18% larger at the same quality (measured against libx264 medium; see below) |
Baseline |
| Supported codecs | H.264, H.265, AV1 (RTX 40 and later) | H.264 |
Prerequisites
You need the following:
- An NVIDIA GPU with NVENC, from the Maxwell generation (GeForce GTX 750, the 900 series and so on) or newer. NVIDIA’s support matrix lists which models support which codecs.
- An NVIDIA driver new enough for your FFmpeg build. The gyan.dev build of FFmpeg 8.1 needs 570.0 or later.
- An FFmpeg build with NVENC. Run
ffmpeg -encoders | grep nvencand check that h264_nvenc and hevc_nvenc are listed. In the Windows Command Prompt, usefindstr nvencinstead ofgrep.
The list only shows what the build contains. It does not prove that your GPU and driver can use NVENC. To check that, encode a one-second test clip. If it finishes without an error, NVENC works:
※ This command requires an NVIDIA GPU environment
ffmpeg -f lavfi -i testsrc2=duration=1 -c:v h264_nvenc -f null -
Basic h264_nvenc Command
The simplest command encodes the video with h264_nvenc and copies the audio as is:
※ This command requires an NVIDIA GPU environment
ffmpeg -i input.mp4 -c:v h264_nvenc -c:a copy output_nvenc.mp4
To set the quality, use -cq. The value runs from 1 to 51, and lower means better quality (0 means automatic):
※ This command requires an NVIDIA GPU environment
ffmpeg -i input.mp4 -c:v h264_nvenc -rc vbr -cq 23 -c:a aac -b:a 128k output_nvenc.mp4
hevc_nvenc (H.265)
H.265 gives a smaller file than H.264 at about the same quality:
※ This command requires an NVIDIA GPU environment
ffmpeg -i input.mp4 -c:v hevc_nvenc -rc vbr -cq 28 -c:a aac -b:a 128k output_hevc_nvenc.mp4
Presets
-preset chooses between speed and quality:
※ This command requires an NVIDIA GPU environment
ffmpeg -i input.mp4 -c:v h264_nvenc -preset p4 -rc vbr -cq 23 output.mp4
There are seven presets, from -preset p1 (fastest) to -preset p7 (best quality). fast, medium, and slow also work. They are other names for p1, p4, and p7, and slow also turns on two-pass encoding.
| Preset | Speed | Quality |
|---|---|---|
| p1 / fast | Fastest | Lower |
| p4 / medium | Balanced | Standard |
| p7 / slow | Slow | High |
Combining GPU Decoding + NVENC Encoding
If the GPU decodes as well, the CPU load drops further:
※ This command requires an NVIDIA GPU environment
ffmpeg -hwaccel cuda -hwaccel_output_format cuda -i input.mp4 -c:v h264_nvenc -preset p4 -rc vbr -cq 23 output.mp4
-hwaccel_output_format cuda keeps the decoded frames in GPU memory and hands them straight to NVENC.
Bitrate-Targeted Encoding
To encode at a set bitrate, use -rc cbr, which keeps the bitrate constant:
※ This command requires an NVIDIA GPU environment
ffmpeg -i input.mp4 -c:v h264_nvenc -rc cbr -b:v 4M -maxrate 4M -bufsize 8M output.mp4
Common Errors and Fixes
No capable devices found
FFmpeg can’t find an NVIDIA GPU that can run NVENC. It may print No CUDA capable devices found instead. Run nvidia-smi to check that the GPU is detected. If the driver is too old, FFmpeg names the version it needs, for example The minimum required Nvidia driver for nvenc is 570.0 or newer.
OpenEncodeSessionEx failed: out of memory (10)
Other processes are already using all the NVENC sessions allowed. On GeForce cards, NVIDIA limits how many NVENC sessions can run at once in the whole system, counting all GeForce cards in it together. The limit depends on the driver and the product line, so look up your card in NVIDIA’s support matrix.
Cannot load libcuda.so.1 (Linux)
FFmpeg can’t find the CUDA library. Look for it with ldconfig -v | grep cuda, or check whether it is in /usr/lib/x86_64-linux-gnu/.
Choosing Between Software and Hardware Encoding
| Use Case | Recommendation |
|---|---|
| Archiving / high quality | libx264 / libx265 (software) |
| Real-time / live streaming | NVENC |
| Batch conversion / time-critical | NVENC |
| Environment without a GPU | Software encoding |
Speed and Quality Ballparks
Speed depends heavily on the GPU generation, the input resolution, and the preset.
- Encode speed: On 60 seconds of live-action 1080p at 24 fps,
h264_nvenc(-preset p4) ran at 434 fps and-preset p7at 279 fps; CPUlibx264(-preset medium) ran at 156 fps, so p4 was about 2.8× faster. For H.265 the gap widens to about 6×:hevc_nvenc(p4) 417 fps againstlibx265(medium) 66 fps. Test setup: RTX 4090, Core i9-14900KF, FFmpeg 8.1, a 10-second Tears of Steel clip (© Blender Foundation, CC BY 3.0) joined into 60 seconds; median of 3 runs each. - Quality: At the same VMAF score (a measure of picture quality),
h264_nvencfiles were 2–5% larger thanlibx264on live action and 11–18% larger on animation (Big Buck Bunny).hevc_nvencagainstlibx265was −2% to +4% on live action and 7–14% on animation (VMAF 85–93; each setting run at 5–6 CRF/CQ values and compared at equal VMAF). The gap depends on the content; on these two clips it was wider on animation. - When to use it: NVENC is 2–6× faster, so it suits streaming, large batch jobs, and work on a deadline.
hevc_nvencat p7 (166 fps) ran about as fast aslibx264(156 fps) and gave files 22–30% smaller at the same quality, which is worth considering when the recipient can play H.265. For an archive you encode once and keep, spending the time onlibx265or SVT-AV1 is also worth considering. The gap depends on the content, quality and preset: on the live-action clip,hevc_nvencp7 came out about the same size aslibx265medium. The measurements are published on the data page.
Common Pitfalls
-preset slow has no effect / presets seem ignored
- Symptom:
-preset slowdoesn’t give the speed and quality you expected. - Cause:
fast,medium, andsloware other names forp1,p4, andp7, and they do work. Butslowalso turns on two-pass encoding (-multipass fullres), so it is even slower thanp7. - Fix: Use
p1–p7so you know exactly what you get, for example-preset p7 -tune hq. Usep6–p7for high quality,p4for a balance, andp1–p2for low latency.
B-frames added but compression doesn’t improve
- Symptom: Adding
-bf 3hardly makes the file smaller, or seems to be ignored. - Cause: B-frames are frames that refer to the frames before and after them, which saves space. NVENC supports them for HEVC only from the Turing generation (RTX 20), so older GPUs may not use HEVC B-frames. H.264 B-frames work on most generations.
- Fix: Check your GPU generation and use
-b_ref_mode middle -bf 3. If HEVC B-frames don’t work on an older GPU, raise the bitrate to keep the quality, or use H.264.
Encoding two or more streams at once fails with out of memory (10)
- Symptom: A batch runs several NVENC processes at once, and the ones over the session limit fail with
OpenEncodeSessionEx failed. - Cause: You hit NVIDIA’s limit on concurrent NVENC sessions. On GeForce cards it is currently 12 for the whole system, counting all GeForce cards in it together, so a second GeForce card does not raise it. Professional cards that NVIDIA lists as “Unrestricted” have no fixed number; the encoder’s capacity and memory set the limit.
- Fix: Look up your GPU’s limit in NVIDIA’s support matrix and run no more encodes at once than that, or update the driver. Writing several outputs from one process can also be more stable.
CPU-decode → GPU-encode isn’t as fast as expected
- Symptom: You switched to NVENC, but CPU usage stays high and the speed doesn’t go up.
- Cause: The CPU still does the decoding. Decoding on the CPU and copying the frames from the CPU to the GPU slow everything down.
- Fix: Put
-hwaccel cuda -hwaccel_output_format cudabefore-i. The GPU then decodes too, and the frames go to NVENC without leaving GPU memory. This alone can leave the decoder threads using the CPU. Encoding 120 seconds of Big Buck Bunny at 1080p and 30 fps withh264_nvenc(p4, CQ 23): CPU decoding ran at 438 fps with 48 s of CPU time;-hwaccel cuda -hwaccel_output_format cudaalone gave 435 fps and 47 s, almost no change; adding-threads 1before-ias well gave 451 fps and 2.7 s (median of 3 runs each). On this PC the speed stayed about the same while CPU use dropped sharply.
Related Articles
- VAAPI (Linux) Hardware Encoding
- VideoToolbox (macOS) Hardware Encoding
- Compress Video — CRF and Target Bitrate
FAQ
How does NVENC compare to libx264 in image quality?
At the same bitrate, NVENC looks slightly worse. The newest NVENC (Ada Lovelace and later) comes close to libx264 at -preset medium, and given its speed it is good enough in practice.
NVENC isn’t working
Check that the GPU is an NVIDIA card, the driver is up to date, and your FFmpeg was built with --enable-nvenc. ffmpeg -encoders | grep nvenc shows whether the build has NVENC.
Should I use CRF or CQ?
Use -cq with NVENC (-crf is for libx264). The value runs from 1 to 51 (0 means automatic). As with libx264’s CRF, lower means better quality. Values of 19–23 are common.
Should I use B-frames?
Yes. Turning on B-frames with -b_ref_mode middle -bf 3 compresses much better. HEVC B-frames need a Turing-generation GPU or newer, so they may not work on older cards.
Can I do streaming delivery with NVENC?
Yes, it works well for low-latency streaming. For the lowest latency, combine -tune ll or -tune ull (ultra-low latency) with -preset p1.