This is my second attempt at fair isle. While it fits way better than my first fair isle hat, my gauge is still way off for the actual color work part. Any tips for tightening my gauge without puckering the colorwork would be appreciated.

I was happy to find a cute hat to use my kool-aid dyed yarn with, but one side of the pattern definitely turned out better than the other. The better side is the one shown in the photographs.

I shortened the brim by 2 rows and k2tog on every row of the decrease - this hat was already getting pretty big!