So, I got some wisdom regarding this. Ask yourself: what are commonly the longest-lasting, most mature relationships between Japanese women and western men you'll ever see?
The answer is US Military Servicemen and Japanese women. Do they speak Japanese? No. Do they know about the culture? No. Do they even want to be there? no, they are deployed often without any say in the matter. But, the relationships thrive and often turn into life-long loving marriages far more than your JET/Exchange Student types.
The harsh, devastating reality about anybody who learns the language is that if you want to be socially popular with Japanese folks, speaking Japanese is not the way. Speaking English is.
Maybe things have changed in the decade since, maybe not, but when I was a student, the Japanese 100 level students (Freshman 1st year) had tons of Japanese friends, but the 400 level students? They didn't have hardly any.
The way I see it, when you speak English, you exude confidence. You appear exotic, interesting, and exciting. Being with you makes them feel smart and worldly, and makes them feel good about themselves. When you speak Japanese, you just sound like a kid with a child's intellect. Being with you makes them feel like babysitters. When she says your Japanese is 'cute,' she means like a baby, not like a grown sexy man... it's not a good omen that continuing to speak Japanese will take you where you want to go.
People will disagree, as I might have too back when I was in that world as a young man, but this is what I have learned. If you DO speak Japanese, remember this: Speaking polite Japanese (masudesu) is always better. It'll take you much further than trying to be 'cool' or 'hip.' It forces people to respect you and take your words in the best possible context. Good luck, my dude.