The thyroid gland starts developing in the pharynx in the foetus/fetus and descends to its final position taking a path through the tongue, hyoid bone and neck muscles. The connection between its original position and its final position is the thyroglossal duct. This duct normally atrophies and closes off before birth but can remain open in some people.
Failure of the duct to atrophy can lead to thyroglossal cysts which can become infected (uncommon but definitely not rare) and thyroglossal duct carcinoma (very rare). In the case of thyroglossal duct carcinoma, the cancerous cells are ectopic thyroid tissue that has been deposited along the thyroglossal duct and usually follows exposure to radiation.