The seismological properties of Earth’s inner core are key to understanding its composition, dynamics, and growth history. Within the inner core, fine-scale heterogeneity has previously been identified from backscattering of high-frequency compressional waves. Here, we use historical earthquake and explosion data from the Large Aperture Seismic Array, USA between 1969 and 1975 to build a 3D map of heterogeneity from the inner core boundary to 500 km depth and determine the geographical distribution of the scatterers across the 40% of the inner core that is visible to the array. Our model has two regions of strong scattering, one beneath eastern Asia and the other beneath South America, both located where past local surveys have identified scattering structure. We suggest that these loci of strong, fine-scale heterogeneities may be related to random alignments of small, inner core crystals due to fast freezing. These areas, which have been identified as having high attenuation and lie beneath colder areas of the core-mantle boundary, potentially provide constraints on the dynamics of the inner core and the motions in the outer core with downwelling in the mantle and outer core possibly associated with strong scattering and inner core heterogeneity.