Tuxedo 17 (en Ubuntu 24) + GIGABYTE AORUS RTX 5060 Ti AI Box Carte Graphique - 16GB GDDR7.
-
Nouveau test : Qwen3.6-35B-A3B-UD-Q4_K_M.gguf => crash.
# llama-bench -m /models/Qwen3.6-35B-A3B-UD-Q4_K_M.gguf ggml_cuda_init: found 2 CUDA devices (Total VRAM: 21656 MiB): Device 0: NVIDIA GeForce RTX 5060 Ti, compute capability 12.0, VMM: yes, VRAM: 15849 MiB Device 1: NVIDIA GeForce RTX 3060 Laptop GPU, compute capability 8.6, VMM: yes, VRAM: 5806 MiB | model | size | params | backend | ngl | test | t/s | | ------------------------------ | ---------: | ---------: | ---------- | --: | --------------: | -------------------: | /home/arias/Documents/llama.cpp/ggml/src/ggml-cuda/ggml-cuda.cu:104: CUDA errorModèle Qwen3.6-27B-Q4_K_M.gguf : OK.
# llama-bench -m /models/Qwen3.6-27B-Q4_K_M.gguf ggml_cuda_init: found 2 CUDA devices (Total VRAM: 21656 MiB): Device 0: NVIDIA GeForce RTX 5060 Ti, compute capability 12.0, VMM: yes, VRAM: 15849 MiB Device 1: NVIDIA GeForce RTX 3060 Laptop GPU, compute capability 8.6, VMM: yes, VRAM: 5806 MiB | model | size | params | backend | ngl | test | t/s | | ------------------------------ | ---------: | ---------: | ---------- | --: | --------------: | -------------------: | | qwen35 27B Q4_K - Medium | 15.65 GiB | 26.90 B | CUDA | -1 | pp512 | 760.71 ± 4.35 | | qwen35 27B Q4_K - Medium | 15.65 GiB | 26.90 B | CUDA | -1 | tg128 | 21.50 ± 0.01 | build: c3d47e696 (10030) -
Test unitaire de carte :
# llama-bench --list-devices ggml_cuda_init: found 2 CUDA devices (Total VRAM: 21692 MiB): Device 0: NVIDIA GeForce RTX 5060 Ti, compute capability 12.0, VMM: yes, VRAM: 15888 MiB Device 1: NVIDIA GeForce RTX 3060 Laptop GPU, compute capability 8.6, VMM: yes, VRAM: 5803 MiB Available devices: CUDA0: NVIDIA GeForce RTX 5060 Ti (15888 MiB, 15752 MiB free) CUDA1: NVIDIA GeForce RTX 3060 Laptop GPU (5803 MiB, 5685 MiB free) # llama-bench -m /models/gemma-3-1b-it-q4_k_m.gguf -dev CUDA0 ggml_cuda_init: found 2 CUDA devices (Total VRAM: 21692 MiB): Device 0: NVIDIA GeForce RTX 5060 Ti, compute capability 12.0, VMM: yes, VRAM: 15888 MiB Device 1: NVIDIA GeForce RTX 3060 Laptop GPU, compute capability 8.6, VMM: yes, VRAM: 5803 MiB | model | size | params | backend | ngl | dev | test | t/s | | ------------------------------ | ---------: | ---------: | ---------- | --: | ------------ | --------------: | -------------------: | | gemma3 1B Q4_K - Medium | 762.49 MiB | 999.89 M | CUDA | -1 | CUDA0 | pp512 | 19952.89 ± 1896.43 | | gemma3 1B Q4_K - Medium | 762.49 MiB | 999.89 M | CUDA | -1 | CUDA0 | tg128 | 268.47 ± 0.43 | build: c3d47e696 (10030) # llama-bench -m /models/gemma-3-1b-it-q4_k_m.gguf -dev CUDA1 ggml_cuda_init: found 2 CUDA devices (Total VRAM: 21692 MiB): Device 0: NVIDIA GeForce RTX 5060 Ti, compute capability 12.0, VMM: yes, VRAM: 15888 MiB Device 1: NVIDIA GeForce RTX 3060 Laptop GPU, compute capability 8.6, VMM: yes, VRAM: 5803 MiB | model | size | params | backend | ngl | dev | test | t/s | | ------------------------------ | ---------: | ---------: | ---------- | --: | ------------ | --------------: | -------------------: | | gemma3 1B Q4_K - Medium | 762.49 MiB | 999.89 M | CUDA | -1 | CUDA1 | pp512 | 11023.49 ± 568.09 | | gemma3 1B Q4_K - Medium | 762.49 MiB | 999.89 M | CUDA | -1 | CUDA1 | tg128 | 229.36 ± 0.40 | build: c3d47e696 (10030) # llama-bench -m /models/gemma-3-1b-it-q4_k_m.gguf -dev CUDA0,CUDA1 ggml_cuda_init: found 2 CUDA devices (Total VRAM: 21692 MiB): Device 0: NVIDIA GeForce RTX 5060 Ti, compute capability 12.0, VMM: yes, VRAM: 15888 MiB Device 1: NVIDIA GeForce RTX 3060 Laptop GPU, compute capability 8.6, VMM: yes, VRAM: 5803 MiB | model | size | params | backend | ngl | dev | test | t/s | | ------------------------------ | ---------: | ---------: | ---------- | --: | ------------ | --------------: | -------------------: | | gemma3 1B Q4_K - Medium | 762.49 MiB | 999.89 M | CUDA | -1 | CUDA0 | pp512 | 19994.19 ± 1751.94 | | gemma3 1B Q4_K - Medium | 762.49 MiB | 999.89 M | CUDA | -1 | CUDA0 | tg128 | 268.49 ± 0.38 | | gemma3 1B Q4_K - Medium | 762.49 MiB | 999.89 M | CUDA | -1 | CUDA1 | pp512 | 11158.41 ± 527.54 | | gemma3 1B Q4_K - Medium | 762.49 MiB | 999.89 M | CUDA | -1 | CUDA1 | tg128 | 229.10 ± 0.37 | build: c3d47e696 (10030) # llama-bench -m /models/gemma-3-1b-it-q4_k_m.gguf ggml_cuda_init: found 2 CUDA devices (Total VRAM: 21692 MiB): Device 0: NVIDIA GeForce RTX 5060 Ti, compute capability 12.0, VMM: yes, VRAM: 15888 MiB Device 1: NVIDIA GeForce RTX 3060 Laptop GPU, compute capability 8.6, VMM: yes, VRAM: 5803 MiB | model | size | params | backend | ngl | test | t/s | | ------------------------------ | ---------: | ---------: | ---------- | --: | --------------: | -------------------: | | gemma3 1B Q4_K - Medium | 762.49 MiB | 999.89 M | CUDA | -1 | pp512 | 16582.32 ± 1005.41 | | gemma3 1B Q4_K - Medium | 762.49 MiB | 999.89 M | CUDA | -1 | tg128 | 261.64 ± 1.81 | build: c3d47e696 (10030) -
Test :
# du -sh /models/* 769M /models/gemma-3-1b-it-q4_k_m.gguf 889M /models/mmproj-BF16(1).gguf 862M /models/mmproj-BF16.gguf 885M /models/mmproj-F16.gguf 5,3G /models/Qwen3.5-9B-Q4_K_M.gguf 16G /models/Qwen3.6-27B-Q4_K_M.gguf 21G /models/Qwen3.6-35B-A3B-UD-Q4_K_M.gguf # llama-bench -m /models/Qwen3.5-9B-Q4_K_M.gguf -dev CUDA0 ggml_cuda_init: found 2 CUDA devices (Total VRAM: 21692 MiB): Device 0: NVIDIA GeForce RTX 5060 Ti, compute capability 12.0, VMM: yes, VRAM: 15888 MiB Device 1: NVIDIA GeForce RTX 3060 Laptop GPU, compute capability 8.6, VMM: yes, VRAM: 5803 MiB | model | size | params | backend | ngl | dev | test | t/s | | ------------------------------ | ---------: | ---------: | ---------- | --: | ------------ | --------------: | -------------------: | | qwen35 9B Q4_K - Medium | 5.28 GiB | 8.95 B | CUDA | -1 | CUDA0 | pp512 | 3041.38 ± 47.16 | | qwen35 9B Q4_K - Medium | 5.28 GiB | 8.95 B | CUDA | -1 | CUDA0 | tg128 | 70.57 ± 0.04 | build: c3d47e696 (10030) # llama-bench -m /models/Qwen3.6-27B-Q4_K_M.gguf -dev CUDA0 ggml_cuda_init: found 2 CUDA devices (Total VRAM: 21692 MiB): Device 0: NVIDIA GeForce RTX 5060 Ti, compute capability 12.0, VMM: yes, VRAM: 15888 MiB Device 1: NVIDIA GeForce RTX 3060 Laptop GPU, compute capability 8.6, VMM: yes, VRAM: 5803 MiB | model | size | params | backend | ngl | dev | test | t/s | | ------------------------------ | ---------: | ---------: | ---------- | --: | ------------ | --------------: | -------------------: | llama_bench: error: failed to create context with model '/models/Qwen3.6-27B-Q4_K_M.gguf' -
-
Installation d’outils pour Hermes Agent IA :
# apt-get install ripgrep # apt-get install chromium-browser # npx playwright install-deps chromium -
Nouveau test avec /models/Qwen3.6-27B-Q3_K_M.gguf :
# llama-bench -m /models/Qwen3.6-27B-Q3_K_M.gguf -dev CUDA1 ggml_cuda_init: found 2 CUDA devices (Total VRAM: 21692 MiB): Device 0: NVIDIA GeForce RTX 5060 Ti, compute capability 12.0, VMM: yes, VRAM: 15888 MiB Device 1: NVIDIA GeForce RTX 3060 Laptop GPU, compute capability 8.6, VMM: yes, VRAM: 5803 MiB | model | size | params | backend | ngl | dev | test | t/s | | ------------------------------ | ---------: | ---------: | ---------- | --: | ------------ | --------------: | -------------------: | llama_bench: error: failed to load model '/models/Qwen3.6-27B-Q3_K_M.gguf' # llama-bench -m /models/Qwen3.6-27B-Q3_K_M.gguf ggml_cuda_init: found 2 CUDA devices (Total VRAM: 21692 MiB): Device 0: NVIDIA GeForce RTX 5060 Ti, compute capability 12.0, VMM: yes, VRAM: 15888 MiB Device 1: NVIDIA GeForce RTX 3060 Laptop GPU, compute capability 8.6, VMM: yes, VRAM: 5803 MiB | model | size | params | backend | ngl | test | t/s | | ------------------------------ | ---------: | ---------: | ---------- | --: | --------------: | -------------------: | | qwen35 27B Q3_K - Medium | 12.64 GiB | 26.90 B | CUDA | -1 | pp512 | 707.38 ± 5.11 | | qwen35 27B Q3_K - Medium | 12.64 GiB | 26.90 B | CUDA | -1 | tg128 | 18.31 ± 0.01 | build: c3d47e696 (10030) # llama-bench -m /models/Qwen3.6-27B-Q3_K_M.gguf -dev CUDA0 ggml_cuda_init: found 2 CUDA devices (Total VRAM: 21692 MiB): Device 0: NVIDIA GeForce RTX 5060 Ti, compute capability 12.0, VMM: yes, VRAM: 15888 MiB Device 1: NVIDIA GeForce RTX 3060 Laptop GPU, compute capability 8.6, VMM: yes, VRAM: 5803 MiB | model | size | params | backend | ngl | dev | test | t/s | | ------------------------------ | ---------: | ---------: | ---------- | --: | ------------ | --------------: | -------------------: | | qwen35 27B Q3_K - Medium | 12.64 GiB | 26.90 B | CUDA | -1 | CUDA0 | pp512 | 841.06 ± 9.58 | | qwen35 27B Q3_K - Medium | 12.64 GiB | 26.90 B | CUDA | -1 | CUDA0 | tg128 | 20.50 ± 0.02 | build: c3d47e696 (10030)Pendant le test sur les deux cartes j’ai lancé gpustat , j’ai les deux cartes qui tournent :
# gpustat tuxedo-17 Fri Jul 17 17:14:51 2026 610.43.03 [0] NVIDIA GeForce RTX 3060 Laptop GPU | 58°C, 40 % | 4156 / 6144 MB | root(4148M) [1] NVIDIA GeForce RTX 5060 Ti | 62°C, 60 % | 8916 / 16311 MB | root(8908M)Quand je suis sur CUDA0 :
gpustat tuxedo-17 Fri Jul 17 17:15:48 2026 610.43.03 [0] NVIDIA GeForce RTX 3060 Laptop GPU | 49°C, 0 % | 2 / 6144 MB | [1] NVIDIA GeForce RTX 5060 Ti | 68°C, 100 % | 12882 / 16311 MB | root(12874M) -
Nouveau test :
# llama-bench -m /models/Qwen3.6-27B-Q3_K_M.gguf -dev CUDA0 -fa on --threads 12 -ngl 999 --cache-type-v q8_0 ggml_cuda_init: found 2 CUDA devices (Total VRAM: 21692 MiB): Device 0: NVIDIA GeForce RTX 5060 Ti, compute capability 12.0, VMM: yes, VRAM: 15888 MiB Device 1: NVIDIA GeForce RTX 3060 Laptop GPU, compute capability 8.6, VMM: yes, VRAM: 5803 MiB | model | size | params | backend | ngl | threads | type_v | fa | dev | test | t/s | | ------------------------------ | ---------: | ---------: | ---------- | --: | ------: | -----: | --: | ------------ | --------------: | -------------------: | | qwen35 27B Q3_K - Medium | 12.64 GiB | 26.90 B | CUDA | 999 | 12 | q8_0 | 1 | CUDA0 | pp512 | 387.17 ± 2.25 | | qwen35 27B Q3_K - Medium | 12.64 GiB | 26.90 B | CUDA | 999 | 12 | q8_0 | 1 | CUDA0 | tg128 | 18.31 ± 0.01 | # llama-bench -m /models/Qwen3.6-27B-Q3_K_M.gguf -fa on --threads 12 -ngl 999 --cache-type-v q8_0 ggml_cuda_init: found 2 CUDA devices (Total VRAM: 21692 MiB): Device 0: NVIDIA GeForce RTX 5060 Ti, compute capability 12.0, VMM: yes, VRAM: 15888 MiB Device 1: NVIDIA GeForce RTX 3060 Laptop GPU, compute capability 8.6, VMM: yes, VRAM: 5803 MiB | model | size | params | backend | ngl | threads | type_v | fa | test | t/s | | ------------------------------ | ---------: | ---------: | ---------- | --: | ------: | -----: | --: | --------------: | -------------------: | | qwen35 27B Q3_K - Medium | 12.64 GiB | 26.90 B | CUDA | 999 | 12 | q8_0 | 1 | pp512 | 367.28 ± 1.88 | | qwen35 27B Q3_K - Medium | 12.64 GiB | 26.90 B | CUDA | 999 | 12 | q8_0 | 1 | tg128 | 16.71 ± 0.03 |
Bonjour ! Vous semblez intéressé par cette conversation, mais vous n’avez pas encore de compte.
Marre de refaire défiler les mêmes messages ? Créez un compte pour retrouver votre position, recevoir des notifications des nouvelles réponses, sauvegarder vos favoris et voter pour les messages que vous appréciez.
Grâce à votre participation, ce message peut devenir encore meilleur 💗
S'inscrire Se connecter