I haven't played the game for a month and a bit so they may have adjusted this in a patch, but originally changing the difficulty did effect xp gain. On harder difficulties the game would scale low level content closer to your actual level resulting in you playing higher level content and so more xp gained. You might also have got more level appropriate loot drops but I never tested that, that's just me theory crafting. Their was this cheeky exploit when you did a quest you could jack the difficulty down to easy during it and when you had to turn it in turn it up to nightmare and you would get a higher xp reward.
Yeah turning the difficulty up gave you way more XP... I hope they didn't patch that out.