Crawley boss Drummy admits: 'Stevenage looked like the home side and we looked like the away side.'
Reds' four-game winning run at home was ended in frustrating style when a 94th minute goal by Matt Godden inflicted defeat.
Town had taken the lead through James Collins from the penalty spot, but for the second game in a row they failed to capitalise on their topscorer finding the net.
Advertisement
Hide AdAdvertisement
Hide AdTwo goals by Godden in the last 14 minute earned Stevenage a rare away victory.
Drummy said: "I'm annoyed, I'm disappointed, it's not what I want.
"Stevenage are a very good side, they time their runs and we have got big players who have got to match them and defend well, which wasn't the way the match ended.
"They were on the front-foot and looked like the home side and we looked like we were the away side.
"We were chasing the ball all-game.
Advertisement
Hide AdAdvertisement
Hide Ad"We are disappointed in our performance, the players know it, it's not the way I want to play our football.
"We've had two bad performances and we've also had many good performances and we've got to get back to the good performances.
"We've got to sit down as a group, regroup and ask why.
"We have got to make everyone understands we are all in this together and it's not all down to one or two individuals, it's the coaching staff as well as to how we perform."