I think I found the problem, and I never would of guessed it was related to the other issue with the bike, that I thought was minor.
The shifter action was really tight, so I thought I would pull the shifter rod and see what the problem was.
When I pulled the side case to remove the clutch, I noticed that there was hardly any clearance between the shifter rod gear sector and the clutch gear. After I removed the clutch, I also noted some metallic sludge immediately below:
I pulled the rod out, and sure enough the rod is bent. I also didn't notice that there is a washer missing that prevents the rod from shifting laterally to the right. Its part #30 here, and it let the interference happen:
My theory is that there was just enough clearance between the gear shift rod-gear sector and the clutch gear for the bike to run at low rpm. When it was in neutral, I could gun the engine as much as possible (the clutch primary gear isn't moving at that point, so no bogging down), but when it was in gear and on the road, the loading of the engine, would cause the gear to start rubbing against the shifter road gear sector. The gears are not straight cut, but have a pitch on them and will thrust a bit.
If you look at my older photo, posted before, that wear wasn't there when I put the engine back together.
Now I have to get a good shifter rod, to see if this is really the problem.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
A bit of background. I got this bike, not running. I purchased it from a farmer (friend of a friend), and it was seized when I got it. It laid in storage in my parent's heated basement for years, and then I finally got to tearing it down for a complete engine rebuild.
Strip down was a bear, because there was cow crap, corn husks and kernels all jammed into the bottom fins of the crankcase (LOL). The previous owner used to ride it in his fields.
When I did the rebuild, most bits seemed good, except for the crank which I had rebuilt.
I finally got it together, but a move across country delayed finishing. In those years I had forgotten about the shifter being a tight fit. I think the shifter seals were also bad, because I replaced them. I think what happened is that the previous owner went down on the bike and bent the shifter fork. I have literally under 20 miles on the bike, since the rebuild.
Live and learn....