There has been renewed interest in the Ruddlesden-Popper phase (n=2) of composition Can+1MnnO3 n+1 in the light of recent research that has highlighted the nature of the improper ferroelectric ground state, which arises due to the couplings between specific combinations of MnO6 octahedral rotations and tilts. A fruitful route to control these octahedral degrees of freedom, and hence such desired physical properties, is through chemical substitution on the A–site cation i.e. Ca2−xSrxMnO4 for n =1, and in light of this, we have reinvestigated the chemistry of this solid solution. Here we focus on a common impurity phase observed during this synthesis which has been termed the “alpha-phase” in the literature. We show that this impurity phase is actually comprised mainly of a structure related to Sr7Mn4O15 but is found here with significantly higher Ca substitution than previously believed possible. Sr7Mn4O15 is an interesting structural type in its own right, but has been mainly overlooked to date, exhibiting interesting physics related to low dimensional magnetic ordering and dimer interactions, and we show here that the structural type is a likely candidate for exhibiting a multiferroic ground state. The prospect of being able to tune the lattice and the exchange interactions through further chemical substitution is likely to lead to a renewed interest in this material.