Theoretical study of topological phase transitions in (Bi_1-xIn_x)₂Se₃ and (Bi_1-xSb_x)₂Se₃

We use first-principles calculations to study the phase transition from a topological to a normal insulator with concentration x in (Bi1-xInx)2Se3 and (Bi1-xSbx)2Se3 in the Bi2Se3 crystal structure. The spin-orbital coupling (SOC) strength is similar in In and Sb, which have similar atomic numbers, so that if the topological transitions in (Bi1-xInx)2Se3 and (Bi1-xSbx)2Se3 are purely driven by the decrease of SOC strength, we would expect to see similar critical concentrations xc in the two systems. However, based on our preliminary calculations, xc is much lower in (Bi1-xInx)2Se3 than in (Bi1-xSbx)2Se3, indicating that different mechanisms control the behavior in the two cases. Specifically, in (Bi1-xSbx)2Se3 we find that the phase transition is mostly dominated by the decrease of SOC. However, for (Bi1-xInx)2Se3, the In 5s orbitals also play an important role, both in the phase-transition behavior and in determining the indirect bulk band gap. Finally, we discuss the accuracy of the energy-level position of the In 5s orbitals in (Bi1-xInx)2Se3 as predicted by density-functional theory and more advanced methods.