I don't own an '81, but I have tinkered a bit with my 1982.
I've sort of ordered these based on ease/price. And these are just ideas.
1) Complete tune up. It almost sounds too easy to be true. Replace spark plugs, plug cables, cap, rotor, worn hoses, fluids, brake pads, rotate/balance tires, clean the carburetor (1981 may need different cleaning procedure), and lubricate the grease fittings. This won't result improved dyno numbers, but you'll be amazed how much better the car will drive with some new parts.
2) Exhaust is a great place to start. The only thing you may have to worry about would be an emissions test. If you want to go all out, start at the headers and work your way to the tail pipe. What i love about this, is you aren't really increasing the power out of your motor, you're just getting the hidden power that is already there. Headers + Mufflers will be around $500. I've seen improvements ranging from 3hp to 30 hp.
3) Intake. Now that you have no problem getting the exhaust out of the car, it is time to improve the air coming in. K&N Filters are a good start. They'll last forever and can help a bit. Intake manifold (i know 1981 is a bit different than other years, but i'm sure people here can suggest a suitable manifold). $50 for K&N filter. $200 for an Edlebrock or Holley intake. $?? for 1981 carb.
5) Camshaft / rocker arms. This is a combination of intake and exhaust. More aggressive cam+rockers will increase the amount of air/fuel coming in and exhaust going out. I've known quite a few people that replace the camshaft as their only project. They get quite a difference and the engine looks all original. Roughly $400. Most cams are about $200 and so is a set of rocker arms.
4) You now have air coming in and exhaust coming out. Time to improve cumbustion. Heads are the easiest way to do that. And often, heads will offer the best power-to-dollar ratio of any project. New pistons can also help with that.
5) Rear Differential. This will add zero power to the engine, but it will get the power to the wheels a lot faster. 1982 came with 2.7:1 gears. (i'd bet 1981 was not too different). The downside to this is your gas mileage will suffer. 1982 was a different transmission, but with 2.7:1 gears I run at 1800 rpms at 70 mph. Moving to 3.55:1 gears would bring me to 2300 rpms. My highway gas mileage would go from 22mpg to 17mpg. This depends on if you want to smoke tires or cruise. This is not a do-it-yourself job. GM used shims to align the ring gear, and it can be a pain to get a new one aligned. This is one i'd probably take to a skilled mechanic.
Those are some of things you can do. They range from cheap and easy (air filter) to things you probably won't do yourself (differential). Every car is different, so what works for me may not work as well for you. Most of these can be done for under $1000 each. (Exhaust and Intake combined would probably be around $1,000 if you do it yourself).
Good luck, enjoy the car, and post some pictures!