Same here, I got the Apple silicone case and it is really good but this phone needs to be caseless so I took it off after a few days. Way too nice to put it in one.
I took my case out today to access the sim card, and I have to say that I really appreciate the caseless look. I mean, well duh, but, and I guess that especially in space grey, there is such a raw look to the phone, in a futuristic way, that you feel more after having used it in a case for a while and got used to the slight extra bezels they give.
I still feel like this phone looks and feels finished and whole when in a case, especially an Apple Case designed in tandem with the phone, but when out of it and you can still use that reduced to the bare essential of what makes a phone, that all-screen black glossy slab of metal and glass, luxurious but subdued, is really something special.
Still going back in the case LOL, I'm not taking chances. And I do love my cases too, on their own merit, and also the way I can use them to switch up the look of the phone.