My current 'hypothesis' for my own situation is that the process goes as follows (not necessarily sequential, I usually jump back and forth between 1 and 2, before I deteriorate and fall into 3 (what I used to call 'setback', and what I now call PEM).
1. Baseline functioning (for me this means that I potter about during the day, I can go to the gym 2-3 times a week (if I adapt my effort to the level of slightly fluctuating fatigue), and I have 3 other main activities during the week, 1 art class and 2 times volunteering at a charity shop).
2a. When I do too much, my physical anxiety increases (I feel nervous, chased) first, and when I keep pushing, my head and eyes start to bother me (pressure on forehead, eyes, face), and my brain fog increases. When I notice this, and take a step back, I recover back to 1 quickly, BUT when I keep going ...
2b. I start to get 'hyper' and wired. I cannot yet distinguish well between 1 and 2b. So, if I think I'm back at 1, but I'm actually in 2b, then I'm pushing myself over the edge and end up in...
3. PEM (In sum, feeling weak, profoundly exhausted and sleepy - can't keep my eyes open -, restless, uncomfortable, extreme brain fog and pressure on head and around and behind eyes, and my muscles 'burn' quickly).
So, I hope I can learn to recognise 2b.
I very much recognise what you're saying about being afraid you're in constant PEM. I think I was in the first 18 months or so. I think it's because you can't grasp how little you can actually do, so you keep overdoing it, because you feel that you can't do any less. At least, that's how it felt for me (I remember often thinking: "This is ridiculous! I should be to do X!" (eg, go for an hour walk or meet up with a friend for an hour). Until I accepted that I really couldn't and only then I started resting more and improved slightly.
And yes, my PEM brain definitely triggers anxiety!