Hong Kong utilized an elimination strategy with intermittent use of public health and social measures and increasingly stringent travel regulations to control SARS-CoV-2 transmission. By analyzing >1700 genome sequences representing 17% of confirmed cases from 23-January-2020 to 26-January-2021, we reveal the effects of fluctuating control measures on the evolution and epidemiology of SARS-CoV-2 lineages in Hong Kong. Despite numerous importations, only three introductions were responsible for 90% of locally-acquired cases, two of which circulated cryptically for weeks while less stringent measures were in place. We found that SARS-CoV-2 within-host diversity was most similar among transmission pairs and epidemiological clusters due to a strong transmission bottleneck through which similar genetic background generates similar within-host diversity.