We discuss how to construct a tight binding model Hamiltonan for the simplest possible solid, composed of hydrogen-like atoms. A single orbital per atom is not sufficient because the on-site electron-electron repulsion mixes in higher energy orbitals. The essential physics is captured by a dynamic Hubbard model with one electronic orbital and an auxiliary spin degree of freedom per site. We point out that this physics can lead to a substantial shift in the position and width of electronic energy bands relative to what is predicted by conventional band structure calculations.