People who notice they are not good at reading rhythms usually think that they are not as good at it as other people. But the reality is that reading rhythms is a lot harder than reading music notes for everyone because rhythms happen in time. For notes, you can stop, think, and name the note (if you are allowed to stop and think.) But for rhythms, if you stop, you are already incorrect because the rhythm is gone.
Everyone who is studying reading music needs to learn how to read rhythms. I've seen a lot of people avoid practicing rhythm reading. Or simply they may not be realizing how important it is. If they don't know how important it is, it's easy to neglect that part. I highly recommend students who are learning how to read music to learn rhythms independently away from reading notes.
Read rhythm without notes:
You can pick any music and practice reading rhythms while ignoring the notations. For example, you can tap the rhythm or play the rhythm only on one note. When you do this, you can focus on reading rhythms.
Recognize the time signature:
But before you do this, the most important thing you need to pay attention is the time signature. Don't just jump in and start tapping the rhythm without recognizing the time signature.
Find the pulse:
After understanding the time signature, the next important thing is for you to develop a slow but strong pulse depending on the time signature. Without an accurate steady pulse, you will not succeed in rhythm reading. The pulse is a foundation. You can't put all kinds of rhythm on top of nothing. Everything is going to collapse. The time signature 4/4 will be 1 - 2 - 3 - 4, 3/4 will be 1 - 2 - 3, 6/8 will be 1 - 2 - 3 - 4 - 5 - 6, and so on.
Practice one rhythm patter at a time:
Then, while keeping the steady pulse in you, practice a certain rhythm pattern repetitively until it ingrains in your body before trying a different rhythm pattern. You need to really hear and feel the rhythm pattern while seeing how the rhythm pattern is written. Once it ingrains in you, next time you see the rhythm pattern in a piece of music, you will be able to apply the rhythm pattern that you already learned on the passage immediately.
I have a feeling that you may find practicing independent-rhythm-reading enjoyable. For most people who are struggling with sight-reading, it's not really the notes that are challenging to read. It is the rhythms. Therefore, when the rhythm reading becomes easier, sight-reading music is going to be very easy overall. And you will also be able to pay attention to other things besides rhythms.