Boyfriend uncaring or just me? PLEASE HELP?

Ok so my boyfriend went to rehab and now he is in a recovery house. He has lived there for one month. We have been together for almost four years, lived together for 3 years, except now. I go see him whenever i can because he cannot drive. I go about three times a week and when i go over there all he wants to do is hangout in the living room with his house mates. By the way they all smoke, including him, and i quit over two years ago and it bothers me. I told my boyfriend that i like to hangout in his room most of the time because i like alone time with him and because the smoke kills me, literally, he does not smoke in his room when im there. When he does hangout with me in his room he is always doing something else like messing on the ipod or computer, he rarely pays me any attention. I ask him for sex and its like a marathan to see how fast he can get done, no foreplay or anything. He does not act like he wants to try to be in a good relationship with me, he says im crazy and its just me. I told him normal couples like to spend alone time together, and he says all i want is sex, but that isnt it. I want to spend time together and if sex happens great if not oh well maybe next time. I dont know what the problem is, he always says he wants to be with me, its me that is not sure about being with him because of the way he is treating me. is it right, how should i talk to him? Please help