Personally, I believe you will get the most bang for your buck with a cam change. The problem that many people run into is trying to run too much cam for the rest of their combination. Crucial to cam selection is compression ratio, third member ratio, stall characteristics (if running an auto trans), the ability of the heads to flow, etc. A cam also can be selected which will provide torque in the lower RPM ranges or HP in the higher range. When looking at a torque/hp curve, the two will cross over each other at some RPM. But where they cross over can be controlled by cam selection.
With that said, if I could only make ONE change and one only, I would look at heads. The heads are where you make the real power and torque. As with cam selection, head selection can be a bit tricky too. For instance, if the intake ports are too large for your combination, torque will suffer. On the other hand, if they are too small, HP will suffer. So you have to choose carefully.
I'm going to put a PS in here regarding headers. I mentioned above that head intake ports need to be sized correctly for the specific application. The same is true for headers. In most instances, large tube headers are not a good match for small blocks. Small tube headers will serve better, especially where torque is concerned.
I guess I edited to add that statement because it can't be stressed enough that components have to be matched in order to come up with a combination that really works well and makes a good combo of power and torque. Here in Southern California, it seems that the neighborhood speed shop is nearly a thing of the past. However, if you can locate a good one, they can point you toward a combo that is proven.
Hope that helps a little.
Ron