Well if indeed there were no screws that explains why the top of your door panel had a gap between it and the window.
Assuming you are talking about the power window switch on the passenger door do NOT pry it out from the outside. On the inside of the door panel you will find that there is a spring clip on the left side of the switch and two little plastic tabs on the right side. These two little tabs are what prevent you from prying out the switch from the outside of the door panel. From inside of the door panel, You must push the tab side towards the front of the car (with the handle of a screw driver) while you gently push the switch out from the back. The spring clip will compress allowing the switch to be pushed out from the back.