Boyfriend upset because after a night of drinking, chose not to want to have sex with him... He's pissed...?

After a night of drinking and having a late night on a work night, it got to be late and after a brief interlude of foreplay suggested we just relax and get some rest for the early hour I needed to get to work... he left the bedroom pissed because he wasn't going to get sex, and today still says he's pissed because I shut him down on his advances. He was drunk and after trying to arouse him with no avail, made the suggestion to cuddle and get some sleep. I wonder if he truly feels rejected - even though it is my right as his girlfriend to not summit to sex at a time I don't feel is right. Or if he's frustrated with himself for drinking too much to the point of not getting an erection... I apologized for upsetting him and told him I loved him, without him saying it back - then admitting that he was pissed and didn't feel like saying that he loved me back... I feel hurt, and aggravated that the one time I deny him (for the last 6 months) that I didn't feel like being intimate... what gives??