This is possibly one of the most amazing things I have ever experienced. Central La Paz lies in a valley, or chasm (make no mistake, this "valley" is still at an altitude of 3,500m), surrounded by a high plateau.
Get a cab way before dawn and ask to be dropped off at a panoramic point somewhere halfway up the mountain: the views at sunrise are breathtaking, and due to La Paz's unique geographical location this experience will probably stay with you for a long time.
Obviously you should not do this alone or take valuables.
Ask a cab driver, preferably one that has been recommended by your hotel or hostel, to drop you off at a panorama point that he knows and recommends. This should be about halfway up the mountain, between central La Paz and the city of El Alto at the top