sb.utils.quirks.apply_quirks() causes jit problems when interacting with other libraries with cuda #2896

Describe the bug

On import, this library "applies quirks" automatically here, which causes an error while I'm using transformers on ubuntu / linux / cuda.

Is this your expected behavior? this is causing me to not be able to use speechbrain with transformers, unless I make my own branch and comment out this line

Expected behaviour

Ability to import speechbrain without messing up other jit uses

To Reproduce

import torch
from transformers.models.deberta_v2.modeling_deberta_v2 import make_log_bucket_position

import speechbrain

t = torch.tensor([100], dtype=torch.int64, device="cuda")
make_log_bucket_position(t, 256, 512)

Environment Details

linux
ubuntu
cuda 12.8
torch 2.2.2

Relevant Log Output

RuntimeError: default_program(85): error: more than one instance of overloaded function "fabs" matches the argument list:
            function "fabs(double)" (declared at line 143437 of __nv_nvrtc_builtin_header.h)
            function "fabs(float)" (declared at line 144453 of __nv_nvrtc_builtin_header.h)
            argument types are: (int64_t)
          int64_t n3 = fabs(n2);
                       ^

Additional Context

No response