Many sound-sensitive dogs become worried when it gets dark in the evening. They quickly associate this time of day and the darkness with the occurrence of fireworks and so their anxiety starts as the light fades. Many of these dogs refuse to go outside at all after dusk.
Dogs are very sensitive and they know the difference between night time and early morning when everything is much quieter, the scents are fresher, and the sun is rising. If your dog is this sensitive please do not force him to go out - act jolly and take him into the garden. Use torches to light it up as much as possible and then allow him to return. Walking him in the dark when he is fearful will only increase the chance of further fear developing.
It would be sensible to seek some advice to deal with his sensitivity to noises, as this is a clear indicator that he is struggling to cope. Your vet will be able to direct you to a qualified behaviourist and your pet insurance may cover the cost.