Pascal support #223
Pascal support
#223
|
The description mentions flash attention which is not supported on Pascal. Does anyone know if flash attention is optional such that it is still possible to run on Pascal class GPUs e.g. P40, P100 or whethet it is required so that these older cards are not supported? |
Answered by
michaelfeil
May 16, 2024
Replies: 1 comment 1 reply
|
Hey @cduk , it used torch s functional F.sdpa. This falls to FA2 or memory efficent attention, or plain MHA depending on your hardware (cpu, cuda compute capability / cache). Pascal therefore is supported. |
1 reply
|
Thanks for the quick reply! TEI says older compute not supported, so maybe this is a selling point for Infinity! |
Answer selected by
cduk
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Hey @cduk ,
it used torch s functional F.sdpa. This falls to FA2 or memory efficent attention, or plain MHA depending on your hardware (cpu, cuda compute capability / cache). Pascal therefore is supported.