I use to have that same trouble. I found out I was putting my roller rockers on wrong.
The natural instinct is to adjust the vavle down, then back the polly lock against the main nut. This will loosen up and fall off.
The trick is two fold. One, clean the nut/polly lock with brake cleaner and dry it off from any free oil.
Add a drop of lock tight to the threads.
I like to follow the GM standard procedure of cold setting by rolling the motor around at top dead center, and 180 out of top dead center, then adjusting the selected valves that are ready at that point. I spin the push rod and tighten the main nut by hand until the push rod stops spinning. "I" (everybody has their one method), then turn the nut 1/2 a turn. At this point I run the polly lock down where it just seats without applying any torque.
**HERE IS THE TRICK** Now, you tighten the polly lock and main adjustment down at the same time. Hold the poly lock in synch with the main nut with your alan wrench. Torque the whole assembly down another 1/4 turn. This gives you a full 3/4 off zero lash and that twin twist in a clock wise direction will lock the polly for good, not to mention the addition of the lock tight.
Hope that helps!
