Heat energy obeys the same laws of conservation as light energy. If a certain substance reflects most light wavelengths, most heat energy will be reflected as well. Therefore, due to the nature of visual light, colors that reflect most wavelengths of light tend to be cooler than those that only reflect a few. Understanding how this principle applies to different colors can allow a person to stay warmer or cooler simply by wearing different colored clothes.
TL;DR (Too Long; Didn't Read)
Dark colors, especially black, absorb more heat since they'll absorb more light from the environment. If you're trying to stay cool, wear light colors, which absorb less heat.
Dark colors absorb a lot more heat than lighter ones because they absorb more light energy. In fact, the closer to black a color is, the more heat it absorbs from light sources. The key is that colors do not absorb different amounts of heat, only heat from light. Dark and light colored clothes coming out of a dryer will be the same temperature. However, because light clothes reflect more light when a person is outside, the accompanying heat from the sun is reflected as well. Since dark clothes reflect little solar light, they reflect little solar heat and are hotter as a result.
Colors like pink or yellow are often called "bright" because of the high degree of light they reflect back. Visual light is composed of numerous different colored wavelengths which make a white light when combined. Therefore light colors such as pastel yellows or pinks are perceived that way because most light wavelengths are reflected back to our eyes. Since most light is reflected, little light (or heat) is absorbed.
While color is the primary factor, other variables can affect how colors absorb heat. Shiny colors are able to reflect significant amounts of light and heat compared to flat colors. Even darker colors can reflect most heat they are exposed to if they have a reflective sheen. Regardless, the heat absorption hierarchy of colors will always remain if all other factors are equal. A shiny deep blue will still absorb more heat than a shiny yellow.
Black and White
Black is the ultimate heat absorber. It absorbs all light on the visual spectrum, creating a void of light. As a result of absorbing all light wavelengths, black is the hottest possible color. White is the opposite. White light is the sum of all wavelengths, so when some people view a white object, they are really viewing all visible light hitting the object's surface and reflecting back. Some heat is still absorbed based on the nature of the object's material, but minimal additional heat is absorbed, making white the coolest possible color.