He seems like a nice fellow, but it's one of the few clear instances where going from child actor to adult actor has been a noticeable and significant downgrade.
I agree with Morrigan about S3 dipping, and S4 taking that dip further. Ivar's actor is quite good, but his character isn't very interesting to me, especially with the inexplicable and ahistoric murder of Sigurd, who seemed like he had potential as an interesting, possibly LGBT character.
I'm in for Lagertha at this point. If/when they kill her, unless the show regains its old quality (unlikely), I might drop it.