Packet Compression was also at full blast during the test. With it off, there would have been zero lag at the end.
We need to find a way to optimized packet compression, and there are still probably some concurrency optimizations to be made.
Thanks everyone!