Well, apparently, not all the sensors on top of lamp posts are photocells.
The red lights are a type of sensor that detects changes in the electric current and if there is anything unusual happening, the red light blinks. Also, it sends a signals to the government agency that in charge—NYCDOP in this case—to request a repair.
Since I moved to NYC I've started walking everywhere and soon noticed a glowing red light on top of public lamp posts. I could not find any pattern on the lamp posts that has it and has not. Also, the red light-emitting device's shape was so irregular so I just thought it as some sort of call out thing.
I guess there were a whole lot of people who look at the sky often and saw that red light. One day I stumbled on to find out the answer to it from one of my random searches; it is a photocell. So, what it does is, it detects the ambient light and knows when to turn on and off the light.
Fun fact: you can actually turn off a lamp post using a laser at night. lol