Interesting issue.
If you are restoring the car and want it to be as "correct" as possible, then technically it should be restored to the way it left the factory. However, I can see the exterior color 'mistake' being a controversial item. Do you plan on having the car judged at a national level? If so, you will have to have some pretty serious documentation to prove that it was a factory mistake. If this is the case, you may want to contact someone like Jerry MacNeish to discuss the documentation you have and paint color impact on judging and/or value.
With all that said, I say if you really wanted a black car and the trim tag and POP shows black, why not restore the car to the way you wanted it in the first place.