GPU and unprivileged LXC

Bind GPU to Unprivileged LXC

Tutorials Used:

• TheOrangeOne.net
• YouTube Video using above tutorial
• Mount Storage
• Karvec suggesting libnvidia-encode/decode on guestOS and learned to use -### corresponding with driver version

Proxmox Setup

• Update Proxmox & Reboot
• Install pve-headers & build-essentials
• update-initramfs
• blacklist free nvidia driver
• download official nvidia driver
• chmod +x driver
• ./run install
• install 32-bit also
• ignore other prompts
• edit modules.d files for loading and conf
• create a rules file
• reboot
• verify driver is working with nvidia-smi
• also could try lshw -C display this will list driver it should say 'nvidia'
• Create CT with ubuntu compatible with jellyfin (18, 20, 22 not 22.10 work)
• Do not start CT on creation
• Edit CT conf file with

 1# Allow cgroup access
 2lxc.cgroup.devices.allow: c 195:* rwm
 3lxc.cgroup.devices.allow: c 243:* rwm
 4
 5# Pass through device files
 6lxc.mount.entry: /dev/nvidia0 dev/nvidia0 none bind,optional,create=file
 7lxc.mount.entry: /dev/nvidiactl dev/nvidiactl none bind,optional,create=file
 8lxc.mount.entry: /dev/nvidia-uvm dev/nvidia-uvm none bind,optional,create=file
 9lxc.mount.entry: /dev/nvidia-modeset dev/nvidia-modeset none bind,optional,create=file
10lxc.mount.entry: /dev/nvidia-uvm-tools dev/nvidia-uvm-tools none bind,optional,create=file

My full config:

 1#<div align="center">
 2#  <a href="https%3A//devboer.com">
 3#    <img src="https%3A//github.com/devboer/dev.portfolio/blob/main/assets/images/logo-pve-112-by-112px.png?raw=true" alt="devboer Logo" height="112">
 4#  </a>
 5#
 6#  ## Jellyfin LXC
 7#
 8#
 9#  %F0%9F%8C%8D [External](https%3A//jellyfin.devboer.com) | %F0%9F%8F%A0 [Internal](https%3A//jellyfin.int.devboer.com) | %F0%9F%94%97 [40.20%3A8096](http%3A//10.0.40.20%3A8096) | %F0%9F%93%9A [GitHub](https%3A//github.com/jellyfin/jellyfin) <br/> %F0%9F%93%8E [Documentation](https%3A//jellyfin.org/docs/) | %E2%84%9E [Helper Scripts](https%3A//helper-scripts.com)
10#</div>
11arch: amd64
12cores: 2
13features: nesting=1
14hostname: jellyfin
15memory: 4096
16# Storage Map Points (mp0 is shared with other containers)
17mp0: /mnt/data/shared-media,mp=/opt/shared-media
18mp1: /mnt/data/app-data/jellyfin,mp=/opt/app-data
19nameserver: 1.1.1.1
20net0: name=eth0,bridge=vmbr0,hwaddr=BC:24:11:84:45:8A,ip=dhcp,tag=40,type=veth
21onboot: 1
22ostype: ubuntu
23rootfs: local-lvm:vm-700-disk-0,size=8G
24startup: order=1
25swap: 512
26tags: media;public
27unprivileged: 1
28# Container UID/GID -> Host UID/GID Mapping
29lxc.idmap: u 0 100000 110
30lxc.idmap: u 110 1086 1
31lxc.idmap: u 111 100111 65425
32lxc.idmap: g 0 100000 118
33lxc.idmap: g 118 1086 1
34lxc.idmap: g 119 100119 65417
35# Device Passthrough Permissions
36lxc.cgroup2.devices.allow: c 195:0 rwm
37lxc.cgroup2.devices.allow: c 195:255 rwm
38lxc.cgroup2.devices.allow: c 195:254 rwm
39lxc.cgroup2.devices.allow: c 236:0 rwm
40lxc.cgroup2.devices.allow: c 236:1 rwm
41lxc.cgroup2.devices.allow: c 10:144 rwm
42# Device Mount Points
43lxc.mount.entry: /dev/nvidia0 dev/nvidia0 none bind,optional,create=file
44lxc.mount.entry: /dev/nvidiactl dev/nvidiactl none bind,optional,create=file
45lxc.mount.entry: /dev/nvidia-modeset dev/nvidia-modeset none bind,optional,create=file
46lxc.mount.entry: /dev/nvidia-uvm dev/nvidia-uvm none bind,optional,create=file
47lxc.mount.entry: /dev/nvidia-uvm-tools dev/nvidia-uvm-tools none bind,optional,create=file
48lxc.mount.entry: /dev/nvram dev/nvram none bind,optional,create=file

• Boot Container
• Update & Reboot
• Install nvidia drivers with argument --no-kernel-module as container will use host's kernel modules
• Again ignore messages and install 32-bit compatibilities
• Install nvtop, libnvidia-encode-525 (this will include -decode) ⟵ This needs to match Driver version
• update initramfs -u
• reboot
• Now container and hypervisor are in sync on driver versions.
• Test (verify) nvidia-smi

[!Note]
You can only see processes running from host with command nvidia-smi while on the guest you can use nvtop

These were the steps I used.... to ready container for hardware transcoding.

Jellyfin installation

Steps followed from here here
You can use the script or do the scripts instructions by hand from the steps below script. curl https://repo.jellyfin.org/install-debuntu.sh \| sudo bash or wget -O- https://repo.jellyfin.org/install-debuntu.sh \| sudo bash
I edited out 'sudo' as I was doing this from root.

Some LXC packages that nvidia driver installer suggested…. libglvnd-dev, pkg-config, and libvulkan1
Host modifications are missing in this guide or rather details. I'm sure they are in the links above but… why

As of Mar 31, 2024…. they still match config and on both guest and host have same driver version but I have an error on 'nvidia-smi' command on guest side. Reboot?
crw-rw-rw- 195,254 root root 31 Mar 02:31 /dev/nvidia-modeset crw-rw-rw- 237,0 root root 31 Mar 02:31 /dev/nvidia-uvm crw-rw-rw- 237,1 root root 31 Mar 02:31 /dev/nvidia-uvm-tools crw-rw-rw- 195,0 root root 31 Mar 02:31 /dev/nvidia0 crw-rw-rw- 195,255 root root 31 Mar 02:31 /dev/nvidiactl crw———- 10,144 root root 27 Mar 19:14 /dev/nvram
/dev/nvidia-caps: Permissions Size User Group Date Modified Name cr———— 240,1 root root 31 Mar 02:31 nvidia-cap1 cr—r—r— 240,2 root root 31 Mar 02:31 nvidia-cap2

Decoding Matrix

Translation on Jellyfin Playback NVIDIA NVENC Decoding checks
[ x ] H264
[ x ] HEVC
[ x ] MPEG2
[ ? ] MPEG4
[ x ] VC1
[ - ] VP8
[ x ] VP9
[ - ] AV1
[ ? ] HEVC 10bit
[ x ] VP9 10bit

GPU Matrix Source