I can't speak about vulkan, but I had an old GTX 680 from 2012, that has worked without issue until a year back or so. I was able to get it recognized by nvidia-smi.
Well, in the case of legacy GPUs you are forced to downgrade drivers. In that case, you can no longer use your recent and legacy GPU simultaneously, if that's what you were hoping for.
But if you do go the route of legacy drivers, they work fine.
I've tried enabling Vulkan on my Intel laptop without a dedicated GPU. But that just makes everything slower.
Did you try running it on the CPU only (BLAS)? Or run it just on the faster and more modern GPUs and see what they do, to compare the numbers to some sort of baseline? Or old GPU only, without more modern ones in the mix? I mean I don't really see the point here. Your computer must be splitting everything up and doing most of the compute somewhere else, if you attach a graphics card with only 1GB of VRAM and the model needs about 8GB. And I'm not sure if the added complexity just makes it slower, or whether it adds something to it. And I'm not sure if I'm missing something or if the output doesn't even show how it gets split up, and what gets executed on which GPU.
Is BLAS faster with CPU only than Vulkan with CPU+iGPU? After failing to make work the SYCL backend in llama.cpp apparently because of a Debian driver issue I ended up using the Vulkan backend but after many tests offloadding to the iGPU doesn't seem to make much difference.
Uh, that's a complicated question. I don't know whether BLAS or Vulkan or SyCL are faster on an iGPU. I think I read many different takes on that. And I suppose it probably changed since I last tested it. People are optimizing the code all the time and it probably also depends on the processor generation and things like that. All I can say setting up SyCL is a hassle and requires like 10GB of development libraries. And I didn't see any noticeable improvement in speed. Either I did something wrong or it's not worth it on my computer. And Vulkan made everything slower on my 8th generation laptop's iGPU. But I'm not sure if that applies generally. But I'm currently sticking to the default backend, I believe that's BLAS. But again on KoboldCPP they replaced OpenBLAS with NoBLAS(?) recently and I haven't kept up to date and it's just too many options... 😅 I don't have any good advice. Maybe try all the options and see which is the fastest... Seems to me using the iGPU likely makes it slower, not faster.
A friend of mine has a old Radeon HD 7850 that was not working with Vulkan out of the box.
Since he didn't want to tinker with it, he gave it to me and I will have a look. I've read you can force the new Vulkan Driver on it with some kernel flags.
p3x.de
I can't speak about vulkan, but I had an old GTX 680 from 2012, that has worked without issue until a year back or so. I was able to get it recognized by nvidia-smi.
I had it running using the proprietary drivers, with the instructions from here, using the legacy method:
https://rpmfusion.org/Howto/NVIDIA#Legacy_GeForce_600.2F700
Is that what you did?
PS: When I mean working without issue I mean gaming on it using proton.
Deleted by author
Well, in the case of legacy GPUs you are forced to downgrade drivers. In that case, you can no longer use your recent and legacy GPU simultaneously, if that's what you were hoping for.
But if you do go the route of legacy drivers, they work fine.
Deleted by author
I think you may be able to use a podman container and pass the gpu over. It will for sure be easier than reinstalling .
I've tried enabling Vulkan on my Intel laptop without a dedicated GPU. But that just makes everything slower.
Did you try running it on the CPU only (BLAS)? Or run it just on the faster and more modern GPUs and see what they do, to compare the numbers to some sort of baseline? Or old GPU only, without more modern ones in the mix? I mean I don't really see the point here. Your computer must be splitting everything up and doing most of the compute somewhere else, if you attach a graphics card with only 1GB of VRAM and the model needs about 8GB. And I'm not sure if the added complexity just makes it slower, or whether it adds something to it. And I'm not sure if I'm missing something or if the output doesn't even show how it gets split up, and what gets executed on which GPU.
Is BLAS faster with CPU only than Vulkan with CPU+iGPU? After failing to make work the SYCL backend in llama.cpp apparently because of a Debian driver issue I ended up using the Vulkan backend but after many tests offloadding to the iGPU doesn't seem to make much difference.
Uh, that's a complicated question. I don't know whether BLAS or Vulkan or SyCL are faster on an iGPU. I think I read many different takes on that. And I suppose it probably changed since I last tested it. People are optimizing the code all the time and it probably also depends on the processor generation and things like that. All I can say setting up SyCL is a hassle and requires like 10GB of development libraries. And I didn't see any noticeable improvement in speed. Either I did something wrong or it's not worth it on my computer. And Vulkan made everything slower on my 8th generation laptop's iGPU. But I'm not sure if that applies generally. But I'm currently sticking to the default backend, I believe that's BLAS. But again on KoboldCPP they replaced OpenBLAS with NoBLAS(?) recently and I haven't kept up to date and it's just too many options... 😅 I don't have any good advice. Maybe try all the options and see which is the fastest... Seems to me using the iGPU likely makes it slower, not faster.
My now old GTX 1060 is working pretty well with Vulkan on Pop!_OS. This is on my older desktop.
RT 5700 + Vulkan working great on my newer one.
A friend of mine has a old Radeon HD 7850 that was not working with Vulkan out of the box.
Since he didn't want to tinker with it, he gave it to me and I will have a look. I've read you can force the new Vulkan Driver on it with some kernel flags.
Deleted by author
Here the doc for the record:
https://wiki.archlinux.org/title/AMDGPU#Set_module_parameters_in_kernel_command_line
Note, they forgot to add to also blacklist the radeon module.