Not really sure what they can do. He's got the overwhelming support of his party members, is just elected, the party is polling well. And it's not as if he's throwing down the gauntlet saying "thanks for your concern, but it's no deal and you can all shut up and lump it", he's being conciliatory to them, and it's hard to flounce (not that many would) when someone isn't arguing with you. All they can do is accept it and see what the manifesto says.
I do think Johnson might moderate a bit post 31st of October. Not because he'll want to but everyone is still saying Cummings will have left by then for an operation which will require prolonged recuperation. Once Cummings goes, Johnson will start listening to Lister more which should tone down the rhetoric a wee bit, but it'll get a lot worse before it gets better.