Pinned
Thanks for the shout out!
Despite the late nights, getting this to work with the other inference folks was possibly the most fun that I've had at work. A wonderful mix of Factorio, Tetris and LeetCode.
Incredible work by @_sholtodouglas @CharlieXYChen and @jacobaustin132 to serve this monstrosity.
Many people thought, and still think, that serving these many tokens is crazy
