He said the adapter ring bolts into the motor should go deeper still into the motor and should have washers under them (otherwise the aluminum will break up under the bolt head when torqued down). We should also set the max RPM limit to the same as the original engine's redline, since over-revving a flywheel (especially with our balance problems) can cause it to explode with deadly results. There's an odd lip on the inner mating surface of the transmission, about 5 mil thick by 50 across, we couldn't figure out why it's there. Dave wasn't concerned about the mating surface issues between motor/flywheel and motor/ring pointed out by the F-SAE guys.
We mounted the flywheel to the motor mount and used a dial indicator to measure the trueness of the flywheel.
The mating surface was very square and flat (+/- 1.5 mil), except for a spot eaten away by corrosion. The friction surface was out of perpendicular with the shaft by +/-4 mil near the outer edge. This was roughly the same as the rear face of the flywheel (indicating the friction surface and rear faces are paralell with each other, but slightly not perpendicular with the shaft, while the pressure plate mating surface is on a different plane, perpendicular with the shaft. The outer diameter was +/- 2.5 mil, though the distribution was irregular, with the min near 12 o'clock (with the top-dead-center notch on top), and the max at 10 o'clock.
We then removed the flywheel to measure the flywheel mount on the motor shaft, and here's where it gets interesting. The mounting face is only 0.5 mil from perpendicular at the outer edge. Then we moved to measuring the outer diameter of the mating surface (which is not a press-fit like it was a few days ago, don't know if that's from the new chamfer or because we cleaned it off with a paper towel). The O.D. there was virtually dead-on, but we came upon a very significant finding. Applying some pressure to the shaft results in significant deflection, without a scale handy, I'd say about 20 lb = 1 mil (and still half that right next to the bearing). We both pretty much ghasped at this point. The motor shaft was never designed to take a load of any significant mass, it was meant to be coupled to the transmission by a small diameter gear.
The clutch assembly weighs a good 30 lbs, so deflecting the tip of the shaft 1.5 mil (and the shaft inside the motor deflecting up probably twice that) could cause quite a lot of problems at low revs, all that flexing could explain some of the heat. At high revs it's a self-reinforcing error, the heavier side of the flywheel will pull on the shaft, which then pulls more mass to its side of the center of rotation, increasing the imbalance further. This explains why the vibration doesn't appear at low revs, and gets exponentially worse at higher revs. This also explains why the wear in the bell housing goes all the way around... with one side of the flywheel flexing out, it basically increases the effective diameter of the flywheel. The gouges in the bell housing are about 20 mil at their deepest.
Now that we understand the phenomenon, we can quantize it: the sides of the housing where the gouges on both sides are 5 mil deep, the housing is 11" wide, giving an effective flywheel diameter of 11.01", over the nominal flywheel diameter of 10.88. This is a difference of 130 mil!! This means the flywheel is roughly 1/16" off-center at speed.
We are talking motor shaft displacements great enough to grind the rotor against the stator inside the motor. We're also talking enough flexing and side loads to burn out the motor bearings (also possibly explaining the thermal issues). You can also see some discoloration of the bearing, also indicative of the bearings getting overheated.
Compare that dangling motor shaft to the flywheel mount on the stock engine.
Solutions we discussed:
Rebuild the flywheel mount to shorten its cantilever from the motor bearing. This is probably the easiest, but should be combined with one of the flywheel solutions below.
Removing the clutch altogether and using encoders and motor feedback loops to force rev-matching on shifting (basically my auto-synchro shifting idea, which does make the clutch unnecessary) the (big) downside to this is there is nothing to soak up impulse torques from bumps in the road (which have been known to snap motor shafts).
Getting a racing "button" clutch. These are ultra-low mass, small diameter clutches that use multiple friction surfaces to get the same grip force as larger diameter clutches. An example are those at JB Racing. They run in the double kilo-buck range, but perhaps there are some available used. This would be the best solution, and the only one that could potentially allow us to exceed 6,000 rpm.
Cut down and swiss-cheese the existing flywheel. This will absolutely require the re-balancing of the flywheel by pros (which could take a week), but could considerably reduce the mass and rotational inertia. Dave suggested Lindskog Balancing.
Get a lightweight replacement flywheel. These use the same clutch/pressure plate, but are made of aluminum with steel friction surfaces. Probably the least desirable solution, since it won't reduce the weight or diameter all that significantly, but still cost a lot.