Tuxedo 17 (en Ubuntu 24) + GIGABYTE AORUS RTX 5060 Ti AI Box Carte Graphique - 16GB GDDR7.

Tuxedo17

Nouveau test :

# git clone https://github.com/lokmantsui/aorus-5090-egpu.git -b nvidia-610.43.02-ubuntu-5060ti

# cd aorus-5090-egpu

# ./install.sh 
replacing /etc/modprobe.d/nvidia.conf
replacing /etc/modprobe.d/nvidia-graphics-drivers.conf
replacing /etc/modprobe.d/nvidia-graphics-drivers-kms.conf
replacing /etc/modprobe.d/nvidia-power-management.conf
installing /etc/modprobe.d/aorus.conf
replacing /etc/default/grub
installing /usr/local/bin/aorus-bridge
installing /usr/local/bin/aorus-modules
installing /etc/systemd/system/aorus.service
creating directory /etc/systemd/system/nvidia-persistenced.service.d
installing /etc/systemd/system/nvidia-persistenced.service.d/aorus.conf
installing /etc/udev/rules.d/99-aorus-egpu.rules
running update-initramfs -u
update-initramfs: Generating /boot/initrd.img-6.17.0-122035-tuxedo
running grub-mkconfig -o /boot/grub/grub.cfg
Sourcing file `/etc/default/grub'
Sourcing file `/etc/default/grub.d/50-tuxedo-fix-nvidia-preserve-vram-suspend.cfg'
Sourcing file `/etc/default/grub.d/90-tuxedo.cfg'
Sourcing file `/etc/default/grub.d/tuxedo-theme.cfg'
Generating grub configuration file ...
Found theme: /usr/share/grub/themes/tuxedo/theme.txt
Found linux image: /boot/vmlinuz-6.17.0-122035-tuxedo
Found initrd image: /boot/initrd.img-6.17.0-122035-tuxedo
Found memtest86+ 64bit EFI image: /boot/memtest86+x64.efi
Warning: os-prober will be executed to detect other bootable partitions.
Its output will be used to detect bootable binaries on them and create new boot entries.
Adding boot menu entry for UEFI Firmware Settings ...
done
reloading systemd manager
enabling aorus.service
Created symlink /etc/systemd/system/multi-user.target.wants/aorus.service → /etc/systemd/system/aorus.service.
reloading udev rules
install complete; reboot required

Tuxedo17

Le script ne voit pas la bonne carte :

# /usr/local/bin/aorus-bridge detect
0000:00:01.0

Je pense qu’il fonctionne seulement quand on a une seule carte.

Je vais donc modifier : /usr/local/bin/aorus-bridge .

J’ai donc :

# lspci -nn | grep -e NVIDIA
01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GA106M [GeForce RTX 3060 Mobile / Max-Q] [10de:2520] (rev a1)
01:00.1 Audio device [0403]: NVIDIA Corporation GA106 High Definition Audio Controller [10de:228e] (rev a1)
05:00.0 VGA compatible controller [0300]: NVIDIA Corporation GB206 [GeForce RTX 5060 Ti] [10de:2d04] (rev a1)
05:00.1 Audio device [0403]: NVIDIA Corporation GB206 High Definition Audio Controller [10de:22eb] (rev a1)

En détail :

# udevadm info -q property -p /sys/bus/pci/devices/0000:05:00.0 
DEVPATH=/devices/pci0000:00/0000:00:07.0/0000:03:00.0/0000:04:00.0/0000:05:00.0
DRIVER=nvidia
PCI_CLASS=30000
PCI_ID=10DE:2D04
PCI_SUBSYS_ID=1458:41CD
PCI_SLOT_NAME=0000:05:00.0
MODALIAS=pci:v000010DEd00002D04sv00001458sd000041CDbc03sc00i00
SUBSYSTEM=pci
USEC_INITIALIZED=2566746846
ID_PCI_CLASS_FROM_DATABASE=Display controller
ID_PCI_SUBCLASS_FROM_DATABASE=VGA compatible controller
ID_PCI_INTERFACE_FROM_DATABASE=VGA controller
ID_VENDOR_FROM_DATABASE=NVIDIA Corporation
ID_PATH=pci-0000:05:00.0
ID_PATH_TAG=pci-0000_05_00_0
ID_FOR_SEAT=pci-pci-0000_05_00_0
TAGS=:seat:master-of-seat:systemd:
CURRENT_TAGS=:master-of-seat:seat:
# udevadm info -q property -p /sys/bus/pci/devices/0000:01:00.0 
DEVPATH=/devices/pci0000:00/0000:00:01.0/0000:01:00.0
DRIVER=nvidia
PCI_CLASS=30000
PCI_ID=10DE:2520
PCI_SUBSYS_ID=1D05:1146
PCI_SLOT_NAME=0000:01:00.0
MODALIAS=pci:v000010DEd00002520sv00001D05sd00001146bc03sc00i00
SUBSYSTEM=pci
USEC_INITIALIZED=1449305
ID_PCI_CLASS_FROM_DATABASE=Display controller
ID_PCI_SUBCLASS_FROM_DATABASE=VGA compatible controller
ID_PCI_INTERFACE_FROM_DATABASE=VGA controller
ID_VENDOR_FROM_DATABASE=NVIDIA Corporation
ID_MODEL_FROM_DATABASE=GA106M [GeForce RTX 3060 Mobile / Max-Q]
ID_PATH=pci-0000:01:00.0
ID_PATH_TAG=pci-0000_01_00_0
ID_FOR_SEAT=pci-pci-0000_01_00_0
TAGS=:systemd:master-of-seat:seat:
CURRENT_TAGS=:master-of-seat:seat:

Je modifier donc modifier :

VENDOR_ID="${VENDOR_ID:-0x10de}"   # NVIDIA
# DEVICE_ID: exact PCI device id to match (e.g. 0x2b85 for RTX 5090). Leave
# empty to match ANY NVIDIA display controller by PCI class — this covers the
# whole RTX 50-series (5090=0x2b85, 5060 Ti=0x2d04, …) without a hardcoded list,
# in the same spirit as is_tb_tunneled below. Set DEVICE_ID to pin one board.
DEVICE_ID="${DEVICE_ID:-0x2d04}"
# PCI base class 0x03 = display controller (VGA 0x0300 / 3D 0x0302); selects the
# GPU function and excludes its HDMI-audio function (class 0x0403).
GPU_CLASS_PREFIX="${GPU_CLASS_PREFIX:-0x03}"

Tuxedo17

Mise à jours du fichier : Fichier : /etc/default/grub

GRUB_DEFAULT=0
GRUB_TIMEOUT_STYLE=hidden
GRUB_TIMEOUT=0
GRUB_DISTRIBUTOR='tuxedo'
GRUB_CMDLINE_LINUX_DEFAULT="quiet splash"
GRUB_CMDLINE_LINUX="pci=resource_alignment iommu.passthrough=1 thunderbolt.host_reset=false pcie_aspm.policy=performance thunderbolt.clx=0 pcie_port_pm=off pci=resource_alignment=35@0000:00:01.0"

Il a du voir la première carte, je change donc :

GRUB_DEFAULT=0
GRUB_TIMEOUT_STYLE=hidden
GRUB_TIMEOUT=0
GRUB_DISTRIBUTOR='tuxedo'
GRUB_CMDLINE_LINUX_DEFAULT="quiet splash"
GRUB_CMDLINE_LINUX="iommu.passthrough=1 thunderbolt.host_reset=false pcie_aspm.policy=performance thunderbolt.clx=0 pcie_port_pm=off pci=resource_alignment=35@0000:00:05.0"

Tuxedo17

Premier bench sans crash :

# nvidia-smi 
Fri Jul 17 15:23:12 2026       
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 595.71.05              Driver Version: 595.71.05      CUDA Version: 13.2     |
+-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 3060 ...    Off |   00000000:01:00.0 Off |                  N/A |
| N/A   49C    P0             28W /  115W |       0MiB /   6144MiB |      2%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+
|   1  NVIDIA GeForce RTX 5060 Ti     Off |   00000000:05:00.0 Off |                  N/A |
| 30%   36C    P0             19W /  180W |       0MiB /  16311MiB |      2%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI              PID   Type   Process name                        GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|  No running processes found                                                             |
+-----------------------------------------------------------------------------------------+

# llama-bench -m /models/gemma-3-1b-it-q4_k_m.gguf 

ggml_cuda_init: found 2 CUDA devices (Total VRAM: 21656 MiB):
  Device 0: NVIDIA GeForce RTX 5060 Ti, compute capability 12.0, VMM: yes, VRAM: 15849 MiB
  Device 1: NVIDIA GeForce RTX 3060 Laptop GPU, compute capability 8.6, VMM: yes, VRAM: 5806 MiB
| model                          |       size |     params | backend    | ngl |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | --------------: | -------------------: |
| gemma3 1B Q4_K - Medium        | 762.49 MiB |   999.89 M | CUDA       |  -1 |           pp512 |    16697.67 ± 675.30 |
| gemma3 1B Q4_K - Medium        | 762.49 MiB |   999.89 M | CUDA       |  -1 |           tg128 |        261.49 ± 2.71 |

build: c3d47e696 (10030)

Donc important :

De modifier /etc/default/grub .
D’utiliser git clone https://github.com/lokmantsui/aorus-5090-egpu.git -b nvidia-610.43.02-ubuntu-5060ti
- De modifier le script /usr/local/bin/aorus-bridge

Bref je dois faire un audit de mon installation.

Tuxedo17

Nouveau test : Qwen3.6-35B-A3B-UD-Q4_K_M.gguf => crash.

# llama-bench -m /models/Qwen3.6-35B-A3B-UD-Q4_K_M.gguf 
ggml_cuda_init: found 2 CUDA devices (Total VRAM: 21656 MiB):
  Device 0: NVIDIA GeForce RTX 5060 Ti, compute capability 12.0, VMM: yes, VRAM: 15849 MiB
  Device 1: NVIDIA GeForce RTX 3060 Laptop GPU, compute capability 8.6, VMM: yes, VRAM: 5806 MiB
| model                          |       size |     params | backend    | ngl |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | --------------: | -------------------: |
/home/arias/Documents/llama.cpp/ggml/src/ggml-cuda/ggml-cuda.cu:104: CUDA error

Modèle Qwen3.6-27B-Q4_K_M.gguf : OK.

# llama-bench -m /models/Qwen3.6-27B-Q4_K_M.gguf 
ggml_cuda_init: found 2 CUDA devices (Total VRAM: 21656 MiB):
  Device 0: NVIDIA GeForce RTX 5060 Ti, compute capability 12.0, VMM: yes, VRAM: 15849 MiB
  Device 1: NVIDIA GeForce RTX 3060 Laptop GPU, compute capability 8.6, VMM: yes, VRAM: 5806 MiB
| model                          |       size |     params | backend    | ngl |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | --------------: | -------------------: |
| qwen35 27B Q4_K - Medium       |  15.65 GiB |    26.90 B | CUDA       |  -1 |           pp512 |        760.71 ± 4.35 |
| qwen35 27B Q4_K - Medium       |  15.65 GiB |    26.90 B | CUDA       |  -1 |           tg128 |         21.50 ± 0.01 |

build: c3d47e696 (10030)

Tuxedo17

Test unitaire de carte :

# llama-bench --list-devices 
ggml_cuda_init: found 2 CUDA devices (Total VRAM: 21692 MiB):
  Device 0: NVIDIA GeForce RTX 5060 Ti, compute capability 12.0, VMM: yes, VRAM: 15888 MiB
  Device 1: NVIDIA GeForce RTX 3060 Laptop GPU, compute capability 8.6, VMM: yes, VRAM: 5803 MiB
Available devices:
  CUDA0: NVIDIA GeForce RTX 5060 Ti (15888 MiB, 15752 MiB free)
  CUDA1: NVIDIA GeForce RTX 3060 Laptop GPU (5803 MiB, 5685 MiB free)


# llama-bench -m /models/gemma-3-1b-it-q4_k_m.gguf -dev CUDA0
ggml_cuda_init: found 2 CUDA devices (Total VRAM: 21692 MiB):
  Device 0: NVIDIA GeForce RTX 5060 Ti, compute capability 12.0, VMM: yes, VRAM: 15888 MiB
  Device 1: NVIDIA GeForce RTX 3060 Laptop GPU, compute capability 8.6, VMM: yes, VRAM: 5803 MiB
| model                          |       size |     params | backend    | ngl | dev          |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | ------------ | --------------: | -------------------: |
| gemma3 1B Q4_K - Medium        | 762.49 MiB |   999.89 M | CUDA       |  -1 | CUDA0        |           pp512 |   19952.89 ± 1896.43 |
| gemma3 1B Q4_K - Medium        | 762.49 MiB |   999.89 M | CUDA       |  -1 | CUDA0        |           tg128 |        268.47 ± 0.43 |

build: c3d47e696 (10030)
# llama-bench -m /models/gemma-3-1b-it-q4_k_m.gguf -dev CUDA1
ggml_cuda_init: found 2 CUDA devices (Total VRAM: 21692 MiB):
  Device 0: NVIDIA GeForce RTX 5060 Ti, compute capability 12.0, VMM: yes, VRAM: 15888 MiB
  Device 1: NVIDIA GeForce RTX 3060 Laptop GPU, compute capability 8.6, VMM: yes, VRAM: 5803 MiB
| model                          |       size |     params | backend    | ngl | dev          |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | ------------ | --------------: | -------------------: |
| gemma3 1B Q4_K - Medium        | 762.49 MiB |   999.89 M | CUDA       |  -1 | CUDA1        |           pp512 |    11023.49 ± 568.09 |
| gemma3 1B Q4_K - Medium        | 762.49 MiB |   999.89 M | CUDA       |  -1 | CUDA1        |           tg128 |        229.36 ± 0.40 |

build: c3d47e696 (10030)
# llama-bench -m /models/gemma-3-1b-it-q4_k_m.gguf -dev CUDA0,CUDA1
ggml_cuda_init: found 2 CUDA devices (Total VRAM: 21692 MiB):
  Device 0: NVIDIA GeForce RTX 5060 Ti, compute capability 12.0, VMM: yes, VRAM: 15888 MiB
  Device 1: NVIDIA GeForce RTX 3060 Laptop GPU, compute capability 8.6, VMM: yes, VRAM: 5803 MiB
| model                          |       size |     params | backend    | ngl | dev          |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | ------------ | --------------: | -------------------: |
| gemma3 1B Q4_K - Medium        | 762.49 MiB |   999.89 M | CUDA       |  -1 | CUDA0        |           pp512 |   19994.19 ± 1751.94 |
| gemma3 1B Q4_K - Medium        | 762.49 MiB |   999.89 M | CUDA       |  -1 | CUDA0        |           tg128 |        268.49 ± 0.38 |
| gemma3 1B Q4_K - Medium        | 762.49 MiB |   999.89 M | CUDA       |  -1 | CUDA1        |           pp512 |    11158.41 ± 527.54 |
| gemma3 1B Q4_K - Medium        | 762.49 MiB |   999.89 M | CUDA       |  -1 | CUDA1        |           tg128 |        229.10 ± 0.37 |

build: c3d47e696 (10030)
# llama-bench -m /models/gemma-3-1b-it-q4_k_m.gguf 
ggml_cuda_init: found 2 CUDA devices (Total VRAM: 21692 MiB):
  Device 0: NVIDIA GeForce RTX 5060 Ti, compute capability 12.0, VMM: yes, VRAM: 15888 MiB
  Device 1: NVIDIA GeForce RTX 3060 Laptop GPU, compute capability 8.6, VMM: yes, VRAM: 5803 MiB
| model                          |       size |     params | backend    | ngl |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | --------------: | -------------------: |
| gemma3 1B Q4_K - Medium        | 762.49 MiB |   999.89 M | CUDA       |  -1 |           pp512 |   16582.32 ± 1005.41 |
| gemma3 1B Q4_K - Medium        | 762.49 MiB |   999.89 M | CUDA       |  -1 |           tg128 |        261.64 ± 1.81 |

build: c3d47e696 (10030)

Tuxedo17

Test :

# du -sh /models/*
769M    /models/gemma-3-1b-it-q4_k_m.gguf
889M    /models/mmproj-BF16(1).gguf
862M    /models/mmproj-BF16.gguf
885M    /models/mmproj-F16.gguf
5,3G    /models/Qwen3.5-9B-Q4_K_M.gguf
16G     /models/Qwen3.6-27B-Q4_K_M.gguf
21G     /models/Qwen3.6-35B-A3B-UD-Q4_K_M.gguf
# llama-bench -m /models/Qwen3.5-9B-Q4_K_M.gguf -dev CUDA0
ggml_cuda_init: found 2 CUDA devices (Total VRAM: 21692 MiB):
  Device 0: NVIDIA GeForce RTX 5060 Ti, compute capability 12.0, VMM: yes, VRAM: 15888 MiB
  Device 1: NVIDIA GeForce RTX 3060 Laptop GPU, compute capability 8.6, VMM: yes, VRAM: 5803 MiB
| model                          |       size |     params | backend    | ngl | dev          |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | ------------ | --------------: | -------------------: |
| qwen35 9B Q4_K - Medium        |   5.28 GiB |     8.95 B | CUDA       |  -1 | CUDA0        |           pp512 |      3041.38 ± 47.16 |
| qwen35 9B Q4_K - Medium        |   5.28 GiB |     8.95 B | CUDA       |  -1 | CUDA0        |           tg128 |         70.57 ± 0.04 |

build: c3d47e696 (10030)
# llama-bench -m /models/Qwen3.6-27B-Q4_K_M.gguf -dev CUDA0
ggml_cuda_init: found 2 CUDA devices (Total VRAM: 21692 MiB):
  Device 0: NVIDIA GeForce RTX 5060 Ti, compute capability 12.0, VMM: yes, VRAM: 15888 MiB
  Device 1: NVIDIA GeForce RTX 3060 Laptop GPU, compute capability 8.6, VMM: yes, VRAM: 5803 MiB
| model                          |       size |     params | backend    | ngl | dev          |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | ------------ | --------------: | -------------------: |
llama_bench: error: failed to create context with model '/models/Qwen3.6-27B-Q4_K_M.gguf'

Tuxedo17

Test à faire avec https://huggingface.co/unsloth/Qwen3.6-27B-GGUF/blob/main/Qwen3.6-27B-Q3_K_M.gguf

Tuxedo17

Installation d’outils pour Hermes Agent IA :

# apt-get install ripgrep
# apt-get install chromium-browser
# npx playwright install-deps chromium

Tuxedo17

Nouveau test avec /models/Qwen3.6-27B-Q3_K_M.gguf :

# llama-bench -m /models/Qwen3.6-27B-Q3_K_M.gguf -dev CUDA1
ggml_cuda_init: found 2 CUDA devices (Total VRAM: 21692 MiB):
  Device 0: NVIDIA GeForce RTX 5060 Ti, compute capability 12.0, VMM: yes, VRAM: 15888 MiB
  Device 1: NVIDIA GeForce RTX 3060 Laptop GPU, compute capability 8.6, VMM: yes, VRAM: 5803 MiB
| model                          |       size |     params | backend    | ngl | dev          |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | ------------ | --------------: | -------------------: |
llama_bench: error: failed to load model '/models/Qwen3.6-27B-Q3_K_M.gguf'

# llama-bench -m /models/Qwen3.6-27B-Q3_K_M.gguf 
ggml_cuda_init: found 2 CUDA devices (Total VRAM: 21692 MiB):
  Device 0: NVIDIA GeForce RTX 5060 Ti, compute capability 12.0, VMM: yes, VRAM: 15888 MiB
  Device 1: NVIDIA GeForce RTX 3060 Laptop GPU, compute capability 8.6, VMM: yes, VRAM: 5803 MiB
| model                          |       size |     params | backend    | ngl |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | --------------: | -------------------: |
| qwen35 27B Q3_K - Medium       |  12.64 GiB |    26.90 B | CUDA       |  -1 |           pp512 |        707.38 ± 5.11 |
| qwen35 27B Q3_K - Medium       |  12.64 GiB |    26.90 B | CUDA       |  -1 |           tg128 |         18.31 ± 0.01 |

build: c3d47e696 (10030)

# llama-bench -m /models/Qwen3.6-27B-Q3_K_M.gguf -dev CUDA0
ggml_cuda_init: found 2 CUDA devices (Total VRAM: 21692 MiB):
  Device 0: NVIDIA GeForce RTX 5060 Ti, compute capability 12.0, VMM: yes, VRAM: 15888 MiB
  Device 1: NVIDIA GeForce RTX 3060 Laptop GPU, compute capability 8.6, VMM: yes, VRAM: 5803 MiB
| model                          |       size |     params | backend    | ngl | dev          |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | ------------ | --------------: | -------------------: |
| qwen35 27B Q3_K - Medium       |  12.64 GiB |    26.90 B | CUDA       |  -1 | CUDA0        |           pp512 |        841.06 ± 9.58 |
| qwen35 27B Q3_K - Medium       |  12.64 GiB |    26.90 B | CUDA       |  -1 | CUDA0        |           tg128 |         20.50 ± 0.02 |

build: c3d47e696 (10030)

Pendant le test sur les deux cartes j’ai lancé gpustat , j’ai les deux cartes qui tournent :

# gpustat 
tuxedo-17                              Fri Jul 17 17:14:51 2026  610.43.03
[0] NVIDIA GeForce RTX 3060 Laptop GPU | 58°C,  40 % |  4156 /  6144 MB | root(4148M)
[1] NVIDIA GeForce RTX 5060 Ti         | 62°C,  60 % |  8916 / 16311 MB | root(8908M)

Quand je suis sur CUDA0 :

 gpustat 
tuxedo-17                              Fri Jul 17 17:15:48 2026  610.43.03
[0] NVIDIA GeForce RTX 3060 Laptop GPU | 49°C,   0 % |     2 /  6144 MB |
[1] NVIDIA GeForce RTX 5060 Ti         | 68°C, 100 % | 12882 / 16311 MB | root(12874M)

Tuxedo17

Nouveau test :

# llama-bench -m /models/Qwen3.6-27B-Q3_K_M.gguf -dev CUDA0 -fa on --threads 12 -ngl 999 --cache-type-v q8_0
ggml_cuda_init: found 2 CUDA devices (Total VRAM: 21692 MiB):
  Device 0: NVIDIA GeForce RTX 5060 Ti, compute capability 12.0, VMM: yes, VRAM: 15888 MiB
  Device 1: NVIDIA GeForce RTX 3060 Laptop GPU, compute capability 8.6, VMM: yes, VRAM: 5803 MiB
| model                          |       size |     params | backend    | ngl | threads | type_v |  fa | dev          |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | ------: | -----: | --: | ------------ | --------------: | -------------------: |
| qwen35 27B Q3_K - Medium       |  12.64 GiB |    26.90 B | CUDA       | 999 |      12 |   q8_0 |   1 | CUDA0        |           pp512 |        387.17 ± 2.25 |
| qwen35 27B Q3_K - Medium       |  12.64 GiB |    26.90 B | CUDA       | 999 |      12 |   q8_0 |   1 | CUDA0        |           tg128 |         18.31 ± 0.01 |

# llama-bench -m /models/Qwen3.6-27B-Q3_K_M.gguf -fa on --threads 12 -ngl 999 --cache-type-v q8_0
ggml_cuda_init: found 2 CUDA devices (Total VRAM: 21692 MiB):
  Device 0: NVIDIA GeForce RTX 5060 Ti, compute capability 12.0, VMM: yes, VRAM: 15888 MiB
  Device 1: NVIDIA GeForce RTX 3060 Laptop GPU, compute capability 8.6, VMM: yes, VRAM: 5803 MiB
| model                          |       size |     params | backend    | ngl | threads | type_v |  fa |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | ------: | -----: | --: | --------------: | -------------------: |
| qwen35 27B Q3_K - Medium       |  12.64 GiB |    26.90 B | CUDA       | 999 |      12 |   q8_0 |   1 |           pp512 |        367.28 ± 1.88 |
| qwen35 27B Q3_K - Medium       |  12.64 GiB |    26.90 B | CUDA       | 999 |      12 |   q8_0 |   1 |           tg128 |         16.71 ± 0.03 |

NodeBB

Tuxedo 17 (en Ubuntu 24) + GIGABYTE AORUS RTX 5060 Ti AI Box Carte Graphique - 16GB GDDR7.