Tuxedo 17 (en Ubuntu 24) + GIGABYTE AORUS RTX 5060 Ti AI Box Carte Graphique - 16GB GDDR7.

farias

Test also :

  nvidia-smi -pm 1                                                                                                                                        
  nvidia-smi -lgc 1000,1000                                                                                                                               
  nvidia-smi -pl 150

farias

Install : https://github.com/lokmantsui/aorus-5090-egpu/tree/nvidia-610.43.02-ubuntu-5060ti

Tuxedo17

Install : https://github.com/apnex/nvidia-driver-injector

sudo git clone https://github.com/apnex/nvidia-driver-injector /root/nvidia-driver-injector
cd /root/nvidia-driver-injector
sudo ./scripts/apply.sh

Tuxedo17

Résultat :

# sudo ./scripts/apply.sh

=== 0/10 conflict check (apnex/aorus-5090-egpu artifacts?) ===
  no aorus-egpu artifacts detected

=== 1/10 kernel cmdline (grubby) ===
  grubby not present (non-Fedora-family host?). Set the following
  kernel cmdline args yourself, by whatever means your distro uses:
    iommu=off
    intel_iommu=off
    thunderbolt.host_reset=false
    pcie_aspm.policy=performance
    thunderbolt.clx=0
    pcie_port_pm=off
  AORUS GPU not currently enumerated; can't auto-set
  pci=resource_alignment. Plug in the eGPU and re-run, or set
  the arg manually using the bridge BDF above your GPU.

=== 2/10 kernel-devel for 6.17.0-122035-tuxedo ===
  kernel-devel present at /lib/modules/6.17.0-122035-tuxedo/build

=== 3/10 gpu UNIX group + GID-rewrite in modprobe.d ===
  gpu group exists (gid=987)

=== 4/10 /etc/modprobe.d/nvidia-driver-injector.conf ===
  installed /etc/modprobe.d/nvidia-driver-injector.conf (NVreg_DeviceFileGID=987)

=== 5/10 nvidia-driver-injector-bridge-link-cap (binary + systemd unit) ===
  bridge-link-cap installed + enabled

=== 6/10 udev rules ===
  udev rules installed (perms + audio-disable)

=== 7/10 Vulkan/EGL/OpenCL ICD disable ===
  /usr/share/vulkan/icd.d/nvidia_icd.x86_64.json not present (driver may not have shipped this ICD)
  /usr/share/vulkan/implicit_layer.d/nvidia_layers.json already disabled
  /usr/share/glvnd/egl_vendor.d/10_nvidia.json already disabled
  /etc/OpenCL/vendors/nvidia.icd already disabled

=== 8/10 apply bridge-link-cap now (without rebooting) ===
  GPU not currently enumerated; cap will apply at next boot

=== 9/10 k3s integration (containerd runtime + RuntimeClass) ===
  k3s not present on this host; skipping k3s integration
  (re-run with k3s installed, or pass --skip-k3s to silence this)

=== 10/10 summary ===
Layer 1 install complete.

Next steps:
  1. Bring up the injector — pick a path:
       # Path A (dev / single-host): docker-compose
       cd /root/nvidia-driver-injector && docker compose up -d

       # Path B (recommended for production): k3s DaemonSet
       cd /root/nvidia-driver-injector && kubectl apply -f k8s/daemonset.yaml
       kubectl rollout status -n kube-system ds/nvidia-driver-injector

  2. Once ready, bring up your GPU consumer (e.g., vLLM):
     - Path A: cd /path/to/workload && docker compose up -d
     - Path B: see docs/consumer-contract.md (nodeSelector +
               runtimeClassName: nvidia)

Tuxedo17

Fichier : /etc/default/grub

GRUB_DEFAULT=0
GRUB_TIMEOUT_STYLE=hidden
GRUB_TIMEOUT=0
GRUB_DISTRIBUTOR='tuxedo'
GRUB_CMDLINE_LINUX_DEFAULT='quiet splash'
GRUB_CMDLINE_LINUX=""

Après

GRUB_DEFAULT=0
GRUB_TIMEOUT_STYLE=hidden
GRUB_TIMEOUT=0
GRUB_DISTRIBUTOR='tuxedo'
GRUB_CMDLINE_LINUX_DEFAULT='quiet splash'
GRUB_CMDLINE_LINUX="iommu=off intel_iommu=off thunderbolt.host_reset=false pcie_aspm.policy=performance thunderbolt.clx=0 pcie_port_pm=off pci=resource_alignment"

Update :

# sudo update-grub
Sourcing file `/etc/default/grub'
Sourcing file `/etc/default/grub.d/50-tuxedo-fix-nvidia-preserve-vram-suspend.cfg'
Sourcing file `/etc/default/grub.d/90-tuxedo.cfg'
Sourcing file `/etc/default/grub.d/tuxedo-theme.cfg'
Generating grub configuration file ...
Found theme: /usr/share/grub/themes/tuxedo/theme.txt
Found linux image: /boot/vmlinuz-6.17.0-122035-tuxedo
Found initrd image: /boot/initrd.img-6.17.0-122035-tuxedo
Found memtest86+ 64bit EFI image: /boot/memtest86+x64.efi
Warning: os-prober will be executed to detect other bootable partitions.
Its output will be used to detect bootable binaries on them and create new boot entries.
Adding boot menu entry for UEFI Firmware Settings ...
done

Tuxedo17

Misère …

# llama-bench -m /models/gemma-3-1b-it-q4_k_m.gguf 
ggml_cuda_init: failed to initialize CUDA: unknown error
| model                          |       size |     params | backend    | ngl |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | --------------: | -------------------: |
| gemma3 1B Q4_K - Medium        | 762.49 MiB |   999.89 M | CUDA       |  -1 |           pp512 |        393.02 ± 6.23 |
| gemma3 1B Q4_K - Medium        | 762.49 MiB |   999.89 M | CUDA       |  -1 |           tg128 |         48.85 ± 1.55 |

build: c3d47e696 (10030)

#  lsmod | grep nvidia_uvm
# modprobe nvidia_uvm
modprobe: ERROR: ../libkmod/libkmod-module.c:1084 command_do() Error running install command '/bin/false' for module nvidia_uvm: retcode 1
modprobe: ERROR: could not insert 'nvidia_uvm': Invalid argument

Tuxedo17

Je vias donc faire une purge :

sudo nvidia-uninstall
sudo apt purge -y '^nvidia-*' '^libnvidia-*'
sudo rm -r /var/lib/dkms/nvidia
sudo apt -y autoremove
sudo update-initramfs -c -k `uname -r`
sudo update-grub2

Et une reinstallation :

# bash ./NVIDIA-Linux-x86_64-610.43.03.run -m=kernel-open

Tuxedo17

Nouveau test :

# git clone https://github.com/lokmantsui/aorus-5090-egpu.git -b nvidia-610.43.02-ubuntu-5060ti

# cd aorus-5090-egpu

# ./install.sh 
replacing /etc/modprobe.d/nvidia.conf
replacing /etc/modprobe.d/nvidia-graphics-drivers.conf
replacing /etc/modprobe.d/nvidia-graphics-drivers-kms.conf
replacing /etc/modprobe.d/nvidia-power-management.conf
installing /etc/modprobe.d/aorus.conf
replacing /etc/default/grub
installing /usr/local/bin/aorus-bridge
installing /usr/local/bin/aorus-modules
installing /etc/systemd/system/aorus.service
creating directory /etc/systemd/system/nvidia-persistenced.service.d
installing /etc/systemd/system/nvidia-persistenced.service.d/aorus.conf
installing /etc/udev/rules.d/99-aorus-egpu.rules
running update-initramfs -u
update-initramfs: Generating /boot/initrd.img-6.17.0-122035-tuxedo
running grub-mkconfig -o /boot/grub/grub.cfg
Sourcing file `/etc/default/grub'
Sourcing file `/etc/default/grub.d/50-tuxedo-fix-nvidia-preserve-vram-suspend.cfg'
Sourcing file `/etc/default/grub.d/90-tuxedo.cfg'
Sourcing file `/etc/default/grub.d/tuxedo-theme.cfg'
Generating grub configuration file ...
Found theme: /usr/share/grub/themes/tuxedo/theme.txt
Found linux image: /boot/vmlinuz-6.17.0-122035-tuxedo
Found initrd image: /boot/initrd.img-6.17.0-122035-tuxedo
Found memtest86+ 64bit EFI image: /boot/memtest86+x64.efi
Warning: os-prober will be executed to detect other bootable partitions.
Its output will be used to detect bootable binaries on them and create new boot entries.
Adding boot menu entry for UEFI Firmware Settings ...
done
reloading systemd manager
enabling aorus.service
Created symlink /etc/systemd/system/multi-user.target.wants/aorus.service → /etc/systemd/system/aorus.service.
reloading udev rules
install complete; reboot required

Tuxedo17

Le script ne voit pas la bonne carte :

# /usr/local/bin/aorus-bridge detect
0000:00:01.0

Je pense qu’il fonctionne seulement quand on a une seule carte.

Je vais donc modifier : /usr/local/bin/aorus-bridge .

J’ai donc :

# lspci -nn | grep -e NVIDIA
01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GA106M [GeForce RTX 3060 Mobile / Max-Q] [10de:2520] (rev a1)
01:00.1 Audio device [0403]: NVIDIA Corporation GA106 High Definition Audio Controller [10de:228e] (rev a1)
05:00.0 VGA compatible controller [0300]: NVIDIA Corporation GB206 [GeForce RTX 5060 Ti] [10de:2d04] (rev a1)
05:00.1 Audio device [0403]: NVIDIA Corporation GB206 High Definition Audio Controller [10de:22eb] (rev a1)

En détail :

# udevadm info -q property -p /sys/bus/pci/devices/0000:05:00.0 
DEVPATH=/devices/pci0000:00/0000:00:07.0/0000:03:00.0/0000:04:00.0/0000:05:00.0
DRIVER=nvidia
PCI_CLASS=30000
PCI_ID=10DE:2D04
PCI_SUBSYS_ID=1458:41CD
PCI_SLOT_NAME=0000:05:00.0
MODALIAS=pci:v000010DEd00002D04sv00001458sd000041CDbc03sc00i00
SUBSYSTEM=pci
USEC_INITIALIZED=2566746846
ID_PCI_CLASS_FROM_DATABASE=Display controller
ID_PCI_SUBCLASS_FROM_DATABASE=VGA compatible controller
ID_PCI_INTERFACE_FROM_DATABASE=VGA controller
ID_VENDOR_FROM_DATABASE=NVIDIA Corporation
ID_PATH=pci-0000:05:00.0
ID_PATH_TAG=pci-0000_05_00_0
ID_FOR_SEAT=pci-pci-0000_05_00_0
TAGS=:seat:master-of-seat:systemd:
CURRENT_TAGS=:master-of-seat:seat:
# udevadm info -q property -p /sys/bus/pci/devices/0000:01:00.0 
DEVPATH=/devices/pci0000:00/0000:00:01.0/0000:01:00.0
DRIVER=nvidia
PCI_CLASS=30000
PCI_ID=10DE:2520
PCI_SUBSYS_ID=1D05:1146
PCI_SLOT_NAME=0000:01:00.0
MODALIAS=pci:v000010DEd00002520sv00001D05sd00001146bc03sc00i00
SUBSYSTEM=pci
USEC_INITIALIZED=1449305
ID_PCI_CLASS_FROM_DATABASE=Display controller
ID_PCI_SUBCLASS_FROM_DATABASE=VGA compatible controller
ID_PCI_INTERFACE_FROM_DATABASE=VGA controller
ID_VENDOR_FROM_DATABASE=NVIDIA Corporation
ID_MODEL_FROM_DATABASE=GA106M [GeForce RTX 3060 Mobile / Max-Q]
ID_PATH=pci-0000:01:00.0
ID_PATH_TAG=pci-0000_01_00_0
ID_FOR_SEAT=pci-pci-0000_01_00_0
TAGS=:systemd:master-of-seat:seat:
CURRENT_TAGS=:master-of-seat:seat:

Je modifier donc modifier :

VENDOR_ID="${VENDOR_ID:-0x10de}"   # NVIDIA
# DEVICE_ID: exact PCI device id to match (e.g. 0x2b85 for RTX 5090). Leave
# empty to match ANY NVIDIA display controller by PCI class — this covers the
# whole RTX 50-series (5090=0x2b85, 5060 Ti=0x2d04, …) without a hardcoded list,
# in the same spirit as is_tb_tunneled below. Set DEVICE_ID to pin one board.
DEVICE_ID="${DEVICE_ID:-0x2d04}"
# PCI base class 0x03 = display controller (VGA 0x0300 / 3D 0x0302); selects the
# GPU function and excludes its HDMI-audio function (class 0x0403).
GPU_CLASS_PREFIX="${GPU_CLASS_PREFIX:-0x03}"

Tuxedo17

Mise à jours du fichier : Fichier : /etc/default/grub

GRUB_DEFAULT=0
GRUB_TIMEOUT_STYLE=hidden
GRUB_TIMEOUT=0
GRUB_DISTRIBUTOR='tuxedo'
GRUB_CMDLINE_LINUX_DEFAULT="quiet splash"
GRUB_CMDLINE_LINUX="pci=resource_alignment iommu.passthrough=1 thunderbolt.host_reset=false pcie_aspm.policy=performance thunderbolt.clx=0 pcie_port_pm=off pci=resource_alignment=35@0000:00:01.0"

Il a du voir la première carte, je change donc :

GRUB_DEFAULT=0
GRUB_TIMEOUT_STYLE=hidden
GRUB_TIMEOUT=0
GRUB_DISTRIBUTOR='tuxedo'
GRUB_CMDLINE_LINUX_DEFAULT="quiet splash"
GRUB_CMDLINE_LINUX="iommu.passthrough=1 thunderbolt.host_reset=false pcie_aspm.policy=performance thunderbolt.clx=0 pcie_port_pm=off pci=resource_alignment=35@0000:00:05.0"

Tuxedo17

Premier bench sans crash :

# nvidia-smi 
Fri Jul 17 15:23:12 2026       
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 595.71.05              Driver Version: 595.71.05      CUDA Version: 13.2     |
+-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 3060 ...    Off |   00000000:01:00.0 Off |                  N/A |
| N/A   49C    P0             28W /  115W |       0MiB /   6144MiB |      2%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+
|   1  NVIDIA GeForce RTX 5060 Ti     Off |   00000000:05:00.0 Off |                  N/A |
| 30%   36C    P0             19W /  180W |       0MiB /  16311MiB |      2%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI              PID   Type   Process name                        GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|  No running processes found                                                             |
+-----------------------------------------------------------------------------------------+

# llama-bench -m /models/gemma-3-1b-it-q4_k_m.gguf 

ggml_cuda_init: found 2 CUDA devices (Total VRAM: 21656 MiB):
  Device 0: NVIDIA GeForce RTX 5060 Ti, compute capability 12.0, VMM: yes, VRAM: 15849 MiB
  Device 1: NVIDIA GeForce RTX 3060 Laptop GPU, compute capability 8.6, VMM: yes, VRAM: 5806 MiB
| model                          |       size |     params | backend    | ngl |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | --------------: | -------------------: |
| gemma3 1B Q4_K - Medium        | 762.49 MiB |   999.89 M | CUDA       |  -1 |           pp512 |    16697.67 ± 675.30 |
| gemma3 1B Q4_K - Medium        | 762.49 MiB |   999.89 M | CUDA       |  -1 |           tg128 |        261.49 ± 2.71 |

build: c3d47e696 (10030)

Donc important :

De modifier /etc/default/grub .
D’utiliser git clone https://github.com/lokmantsui/aorus-5090-egpu.git -b nvidia-610.43.02-ubuntu-5060ti
- De modifier le script /usr/local/bin/aorus-bridge

Bref je dois faire un audit de mon installation.

Tuxedo17

Nouveau test : Qwen3.6-35B-A3B-UD-Q4_K_M.gguf => crash.

# llama-bench -m /models/Qwen3.6-35B-A3B-UD-Q4_K_M.gguf 
ggml_cuda_init: found 2 CUDA devices (Total VRAM: 21656 MiB):
  Device 0: NVIDIA GeForce RTX 5060 Ti, compute capability 12.0, VMM: yes, VRAM: 15849 MiB
  Device 1: NVIDIA GeForce RTX 3060 Laptop GPU, compute capability 8.6, VMM: yes, VRAM: 5806 MiB
| model                          |       size |     params | backend    | ngl |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | --------------: | -------------------: |
/home/arias/Documents/llama.cpp/ggml/src/ggml-cuda/ggml-cuda.cu:104: CUDA error

Modèle Qwen3.6-27B-Q4_K_M.gguf : OK.

# llama-bench -m /models/Qwen3.6-27B-Q4_K_M.gguf 
ggml_cuda_init: found 2 CUDA devices (Total VRAM: 21656 MiB):
  Device 0: NVIDIA GeForce RTX 5060 Ti, compute capability 12.0, VMM: yes, VRAM: 15849 MiB
  Device 1: NVIDIA GeForce RTX 3060 Laptop GPU, compute capability 8.6, VMM: yes, VRAM: 5806 MiB
| model                          |       size |     params | backend    | ngl |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | --------------: | -------------------: |
| qwen35 27B Q4_K - Medium       |  15.65 GiB |    26.90 B | CUDA       |  -1 |           pp512 |        760.71 ± 4.35 |
| qwen35 27B Q4_K - Medium       |  15.65 GiB |    26.90 B | CUDA       |  -1 |           tg128 |         21.50 ± 0.01 |

build: c3d47e696 (10030)

Tuxedo17

Test unitaire de carte :

# llama-bench --list-devices 
ggml_cuda_init: found 2 CUDA devices (Total VRAM: 21692 MiB):
  Device 0: NVIDIA GeForce RTX 5060 Ti, compute capability 12.0, VMM: yes, VRAM: 15888 MiB
  Device 1: NVIDIA GeForce RTX 3060 Laptop GPU, compute capability 8.6, VMM: yes, VRAM: 5803 MiB
Available devices:
  CUDA0: NVIDIA GeForce RTX 5060 Ti (15888 MiB, 15752 MiB free)
  CUDA1: NVIDIA GeForce RTX 3060 Laptop GPU (5803 MiB, 5685 MiB free)


# llama-bench -m /models/gemma-3-1b-it-q4_k_m.gguf -dev CUDA0
ggml_cuda_init: found 2 CUDA devices (Total VRAM: 21692 MiB):
  Device 0: NVIDIA GeForce RTX 5060 Ti, compute capability 12.0, VMM: yes, VRAM: 15888 MiB
  Device 1: NVIDIA GeForce RTX 3060 Laptop GPU, compute capability 8.6, VMM: yes, VRAM: 5803 MiB
| model                          |       size |     params | backend    | ngl | dev          |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | ------------ | --------------: | -------------------: |
| gemma3 1B Q4_K - Medium        | 762.49 MiB |   999.89 M | CUDA       |  -1 | CUDA0        |           pp512 |   19952.89 ± 1896.43 |
| gemma3 1B Q4_K - Medium        | 762.49 MiB |   999.89 M | CUDA       |  -1 | CUDA0        |           tg128 |        268.47 ± 0.43 |

build: c3d47e696 (10030)
# llama-bench -m /models/gemma-3-1b-it-q4_k_m.gguf -dev CUDA1
ggml_cuda_init: found 2 CUDA devices (Total VRAM: 21692 MiB):
  Device 0: NVIDIA GeForce RTX 5060 Ti, compute capability 12.0, VMM: yes, VRAM: 15888 MiB
  Device 1: NVIDIA GeForce RTX 3060 Laptop GPU, compute capability 8.6, VMM: yes, VRAM: 5803 MiB
| model                          |       size |     params | backend    | ngl | dev          |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | ------------ | --------------: | -------------------: |
| gemma3 1B Q4_K - Medium        | 762.49 MiB |   999.89 M | CUDA       |  -1 | CUDA1        |           pp512 |    11023.49 ± 568.09 |
| gemma3 1B Q4_K - Medium        | 762.49 MiB |   999.89 M | CUDA       |  -1 | CUDA1        |           tg128 |        229.36 ± 0.40 |

build: c3d47e696 (10030)
# llama-bench -m /models/gemma-3-1b-it-q4_k_m.gguf -dev CUDA0,CUDA1
ggml_cuda_init: found 2 CUDA devices (Total VRAM: 21692 MiB):
  Device 0: NVIDIA GeForce RTX 5060 Ti, compute capability 12.0, VMM: yes, VRAM: 15888 MiB
  Device 1: NVIDIA GeForce RTX 3060 Laptop GPU, compute capability 8.6, VMM: yes, VRAM: 5803 MiB
| model                          |       size |     params | backend    | ngl | dev          |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | ------------ | --------------: | -------------------: |
| gemma3 1B Q4_K - Medium        | 762.49 MiB |   999.89 M | CUDA       |  -1 | CUDA0        |           pp512 |   19994.19 ± 1751.94 |
| gemma3 1B Q4_K - Medium        | 762.49 MiB |   999.89 M | CUDA       |  -1 | CUDA0        |           tg128 |        268.49 ± 0.38 |
| gemma3 1B Q4_K - Medium        | 762.49 MiB |   999.89 M | CUDA       |  -1 | CUDA1        |           pp512 |    11158.41 ± 527.54 |
| gemma3 1B Q4_K - Medium        | 762.49 MiB |   999.89 M | CUDA       |  -1 | CUDA1        |           tg128 |        229.10 ± 0.37 |

build: c3d47e696 (10030)
# llama-bench -m /models/gemma-3-1b-it-q4_k_m.gguf 
ggml_cuda_init: found 2 CUDA devices (Total VRAM: 21692 MiB):
  Device 0: NVIDIA GeForce RTX 5060 Ti, compute capability 12.0, VMM: yes, VRAM: 15888 MiB
  Device 1: NVIDIA GeForce RTX 3060 Laptop GPU, compute capability 8.6, VMM: yes, VRAM: 5803 MiB
| model                          |       size |     params | backend    | ngl |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | --------------: | -------------------: |
| gemma3 1B Q4_K - Medium        | 762.49 MiB |   999.89 M | CUDA       |  -1 |           pp512 |   16582.32 ± 1005.41 |
| gemma3 1B Q4_K - Medium        | 762.49 MiB |   999.89 M | CUDA       |  -1 |           tg128 |        261.64 ± 1.81 |

build: c3d47e696 (10030)

Tuxedo17

Test :

# du -sh /models/*
769M    /models/gemma-3-1b-it-q4_k_m.gguf
889M    /models/mmproj-BF16(1).gguf
862M    /models/mmproj-BF16.gguf
885M    /models/mmproj-F16.gguf
5,3G    /models/Qwen3.5-9B-Q4_K_M.gguf
16G     /models/Qwen3.6-27B-Q4_K_M.gguf
21G     /models/Qwen3.6-35B-A3B-UD-Q4_K_M.gguf
# llama-bench -m /models/Qwen3.5-9B-Q4_K_M.gguf -dev CUDA0
ggml_cuda_init: found 2 CUDA devices (Total VRAM: 21692 MiB):
  Device 0: NVIDIA GeForce RTX 5060 Ti, compute capability 12.0, VMM: yes, VRAM: 15888 MiB
  Device 1: NVIDIA GeForce RTX 3060 Laptop GPU, compute capability 8.6, VMM: yes, VRAM: 5803 MiB
| model                          |       size |     params | backend    | ngl | dev          |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | ------------ | --------------: | -------------------: |
| qwen35 9B Q4_K - Medium        |   5.28 GiB |     8.95 B | CUDA       |  -1 | CUDA0        |           pp512 |      3041.38 ± 47.16 |
| qwen35 9B Q4_K - Medium        |   5.28 GiB |     8.95 B | CUDA       |  -1 | CUDA0        |           tg128 |         70.57 ± 0.04 |

build: c3d47e696 (10030)
# llama-bench -m /models/Qwen3.6-27B-Q4_K_M.gguf -dev CUDA0
ggml_cuda_init: found 2 CUDA devices (Total VRAM: 21692 MiB):
  Device 0: NVIDIA GeForce RTX 5060 Ti, compute capability 12.0, VMM: yes, VRAM: 15888 MiB
  Device 1: NVIDIA GeForce RTX 3060 Laptop GPU, compute capability 8.6, VMM: yes, VRAM: 5803 MiB
| model                          |       size |     params | backend    | ngl | dev          |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | ------------ | --------------: | -------------------: |
llama_bench: error: failed to create context with model '/models/Qwen3.6-27B-Q4_K_M.gguf'

Tuxedo17

Test à faire avec https://huggingface.co/unsloth/Qwen3.6-27B-GGUF/blob/main/Qwen3.6-27B-Q3_K_M.gguf

Tuxedo17

Installation d’outils pour Hermes Agent IA :

# apt-get install ripgrep
# apt-get install chromium-browser
# npx playwright install-deps chromium

Tuxedo17

Nouveau test avec /models/Qwen3.6-27B-Q3_K_M.gguf :

# llama-bench -m /models/Qwen3.6-27B-Q3_K_M.gguf -dev CUDA1
ggml_cuda_init: found 2 CUDA devices (Total VRAM: 21692 MiB):
  Device 0: NVIDIA GeForce RTX 5060 Ti, compute capability 12.0, VMM: yes, VRAM: 15888 MiB
  Device 1: NVIDIA GeForce RTX 3060 Laptop GPU, compute capability 8.6, VMM: yes, VRAM: 5803 MiB
| model                          |       size |     params | backend    | ngl | dev          |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | ------------ | --------------: | -------------------: |
llama_bench: error: failed to load model '/models/Qwen3.6-27B-Q3_K_M.gguf'

# llama-bench -m /models/Qwen3.6-27B-Q3_K_M.gguf 
ggml_cuda_init: found 2 CUDA devices (Total VRAM: 21692 MiB):
  Device 0: NVIDIA GeForce RTX 5060 Ti, compute capability 12.0, VMM: yes, VRAM: 15888 MiB
  Device 1: NVIDIA GeForce RTX 3060 Laptop GPU, compute capability 8.6, VMM: yes, VRAM: 5803 MiB
| model                          |       size |     params | backend    | ngl |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | --------------: | -------------------: |
| qwen35 27B Q3_K - Medium       |  12.64 GiB |    26.90 B | CUDA       |  -1 |           pp512 |        707.38 ± 5.11 |
| qwen35 27B Q3_K - Medium       |  12.64 GiB |    26.90 B | CUDA       |  -1 |           tg128 |         18.31 ± 0.01 |

build: c3d47e696 (10030)

# llama-bench -m /models/Qwen3.6-27B-Q3_K_M.gguf -dev CUDA0
ggml_cuda_init: found 2 CUDA devices (Total VRAM: 21692 MiB):
  Device 0: NVIDIA GeForce RTX 5060 Ti, compute capability 12.0, VMM: yes, VRAM: 15888 MiB
  Device 1: NVIDIA GeForce RTX 3060 Laptop GPU, compute capability 8.6, VMM: yes, VRAM: 5803 MiB
| model                          |       size |     params | backend    | ngl | dev          |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | ------------ | --------------: | -------------------: |
| qwen35 27B Q3_K - Medium       |  12.64 GiB |    26.90 B | CUDA       |  -1 | CUDA0        |           pp512 |        841.06 ± 9.58 |
| qwen35 27B Q3_K - Medium       |  12.64 GiB |    26.90 B | CUDA       |  -1 | CUDA0        |           tg128 |         20.50 ± 0.02 |

build: c3d47e696 (10030)

Pendant le test sur les deux cartes j’ai lancé gpustat , j’ai les deux cartes qui tournent :

# gpustat 
tuxedo-17                              Fri Jul 17 17:14:51 2026  610.43.03
[0] NVIDIA GeForce RTX 3060 Laptop GPU | 58°C,  40 % |  4156 /  6144 MB | root(4148M)
[1] NVIDIA GeForce RTX 5060 Ti         | 62°C,  60 % |  8916 / 16311 MB | root(8908M)

Quand je suis sur CUDA0 :

 gpustat 
tuxedo-17                              Fri Jul 17 17:15:48 2026  610.43.03
[0] NVIDIA GeForce RTX 3060 Laptop GPU | 49°C,   0 % |     2 /  6144 MB |
[1] NVIDIA GeForce RTX 5060 Ti         | 68°C, 100 % | 12882 / 16311 MB | root(12874M)

Tuxedo17

Nouveau test :

# llama-bench -m /models/Qwen3.6-27B-Q3_K_M.gguf -dev CUDA0 -fa on --threads 12 -ngl 999 --cache-type-v q8_0
ggml_cuda_init: found 2 CUDA devices (Total VRAM: 21692 MiB):
  Device 0: NVIDIA GeForce RTX 5060 Ti, compute capability 12.0, VMM: yes, VRAM: 15888 MiB
  Device 1: NVIDIA GeForce RTX 3060 Laptop GPU, compute capability 8.6, VMM: yes, VRAM: 5803 MiB
| model                          |       size |     params | backend    | ngl | threads | type_v |  fa | dev          |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | ------: | -----: | --: | ------------ | --------------: | -------------------: |
| qwen35 27B Q3_K - Medium       |  12.64 GiB |    26.90 B | CUDA       | 999 |      12 |   q8_0 |   1 | CUDA0        |           pp512 |        387.17 ± 2.25 |
| qwen35 27B Q3_K - Medium       |  12.64 GiB |    26.90 B | CUDA       | 999 |      12 |   q8_0 |   1 | CUDA0        |           tg128 |         18.31 ± 0.01 |

# llama-bench -m /models/Qwen3.6-27B-Q3_K_M.gguf -fa on --threads 12 -ngl 999 --cache-type-v q8_0
ggml_cuda_init: found 2 CUDA devices (Total VRAM: 21692 MiB):
  Device 0: NVIDIA GeForce RTX 5060 Ti, compute capability 12.0, VMM: yes, VRAM: 15888 MiB
  Device 1: NVIDIA GeForce RTX 3060 Laptop GPU, compute capability 8.6, VMM: yes, VRAM: 5803 MiB
| model                          |       size |     params | backend    | ngl | threads | type_v |  fa |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | ------: | -----: | --: | --------------: | -------------------: |
| qwen35 27B Q3_K - Medium       |  12.64 GiB |    26.90 B | CUDA       | 999 |      12 |   q8_0 |   1 |           pp512 |        367.28 ± 1.88 |
| qwen35 27B Q3_K - Medium       |  12.64 GiB |    26.90 B | CUDA       | 999 |      12 |   q8_0 |   1 |           tg128 |         16.71 ± 0.03 |

NodeBB

Tuxedo 17 (en Ubuntu 24) + GIGABYTE AORUS RTX 5060 Ti AI Box Carte Graphique - 16GB GDDR7.