Rotary Positional Embedding: [paper]

Let

We need the transformation functions above ($f_q$ and $f_k$), such that they satisfy the below equation, the resultant function should encode relative position information

Reference-1

Reference-2