It is generally believed that magnetization measurements on sulfur hydride under high pressure performed in 2015 [1] provided "final convincing evidence of superconductivity" [2] in that material, in agreement with theoretical predictions [3,4]. Supported by this precedent, drops in resistance that were later observed in several other hydrides under high pressure [2,5] have been generally accepted as evidence of superconductivity without corroborating evidence from magnetic measurements. In this paper we challenge the original interpretation that the magnetic measurements on sulfur hydride performed in 2015 were evidence of superconductivity. We point out that a large paramagnetic contribution to the magnetic susceptibility was detected below Tc and argue that its temperature dependence rules out the possibility that it would be a background signal; instead the temperature dependence indicates that the paramagnetic behavior originated in the sample. We discuss possible explanations for this remarkable behavior and conclude that standard superconductors would not show such behavior. We also survey all the other published data from magnetic measurements on this class of materials and conclude that they do not provide strong evidence for superconductivity. Consequently, we call into question the generally accepted view that conventional superconductivity in hydrogen-rich materials at high temperature and pressure is a reality, and discuss the implications if it is not.