nvvl - A library that uses hardware acceleration to load sequences of video frames to facilitate machine learning training

  •        16

NVVL (NVIDIA Video Loader) is a library to load random sequences of video frames from compressed video files to facilitate machine learning training. It uses FFmpeg's libraries to parse and read the compressed packets from video files and the video decoding hardware available on NVIDIA GPUs to off-load and accelerate the decoding of those packets, providing a ready-for-training tensor in GPU device memory. NVVL can additionally perform data augmentation while loading the frames. Frames can be scaled, cropped, and flipped horizontally using the GPUs dedicated texture mapping units. Output can be in RGB or YCbCr color space, normalized to [0, 1] or [0, 255], and in float, half, or uint8 tensors. Using compressed video files instead of individual frame image files significantly reduces the demands on the storage and I/O systems during training. Storing video datasets as video files consumes an order of magnitude less disk space, allowing for larger datasets to both fit in system RAM as well as local SSDs for fast access. During loading fewer bytes must be read from disk. Fitting on smaller, faster storage and reading fewer bytes at load time allievates the bottleneck of retrieving data from disks, which will only get worse as GPUs get faster. For the dataset used in our example project, H.264 compressed .mp4 files were nearly 40x smaller than storing frames as .png files.

https://github.com/NVIDIA/nvvl

Tags
Implementation
License
Platform

   




Related Projects

vim-olive - Vim Mode Line Verifier

  •    VimL

Vim Mode Line Verifier

vim-mash - Motion Activated Search Highlighter for Vim

  •    VimL

Motion Activated Search Highlighter for Vim

vim-grillz - Flash your wicked grillz!

  •    VimL

Flash your wicked grillz!

vim-foist - Complete whole lines from any partial therein

  •    VimL

Complete whole lines from any partial therein

vim-efmc - Vim Error Format Compiler

  •    VimL

Vim Error Format Compiler


uzbl-utrs - mkng uzbl tlrbl

  •    Javascript

mkng uzbl tlrbl

toycsv - A toy CSV parser written in ruby + lexr + racc

  •    Ruby

A toy CSV parser written in ruby + lexr + racc

tiktok - TikTok provides a simple asynchronous timer object for VimL.

  •    VimL

TikTok provides a simple asynchronous timer object for VimL.

tabby - Using Vim's Tabs the Right Way

  •    VimL

Using Vim's Tabs the Right Way

SohiVila - GNU source-highlight Vim language

  •    

GNU source-highlight Vim language

SinTax - A DSL for generating Vim syntax highlighting files

  •    VimL

A DSL for generating Vim syntax highlighting files

rkdots - Generate a graphviz visualisation of the given javascript statement.

  •    Ruby

Generate a graphviz visualisation of the given javascript statement.

RelNumBar - Show relativenumbers alongside normal numbers in Vim

  •    VimL

Show relativenumbers alongside normal numbers in Vim

Punisher - Punisher hurts you where it hurts most - your time.

  •    VimL

Punisher hurts you where it hurts most - your time.

noisy - Don't chat quietly (weechat channel noises)

  •    Perl

Don't chat quietly (weechat channel noises)

newlisp-manual - Asciidoc version of the newLISP manual

  •    Common

Asciidoc version of the newLISP manual

Marathon-Wow - Wow Even Longer

  •    Common

Wow Even Longer

Land-of-newLISP - Selected snippets from Land of Lisp rewritten in newLISP

  •    Common

Selected snippets from Land of Lisp rewritten in newLISP

Fossilise - Poor man's collaborative editing in Vim

  •    VimL

Poor man's collaborative editing in Vim

firstly - Convert Between Numeric, Spelt, and Short & Long Ordinal Forms of Numbers

  •    VimL

Convert Between Numeric, Spelt, and Short & Long Ordinal Forms of Numbers