Johnson brushes off conservative pushback, tries to lower expectations