Hemispherical asymmetry is a prominent feature of Earth's inner core, but how this asymmetry relates to core growth is unknown. Based on multiple-scattering modeling of seismic velocity and attenuation measurements sampling the whole uppermost inner core, we propose that the growth of the solid core implies an eastward drift of the material, driven by crystallization in the Western Hemisphere and melting in the Eastern Hemisphere. This self-sustained translational motion generates an asymmetric distribution of sizes of iron crystals, which grow during their translation. The invoked dynamical process is still active today, which supports the idea of a young inner core.