Traditionally bases have a bitter taste while acids are sour, but in chemistry, the definitions have evolved so that substances are either bases or acids using their chemical properties. This classification is important because acids and bases can react to form salts, and they are the basis of many other types of common chemical reactions. Bases have certain chemical properties in common, and the selection of a suitable chemical can affect the reaction's outcome.
TL;DR (Too Long; Didn't Read)
The more limited and older definition was that a base is a substance that dissolves in water and dissociates into a hydroxide or OH- ion and a positive ion. In the more general definition, a base is a substance that when dissolved in water increases the number of hydroxide ions. This definition is broad enough to include substances that themselves do not have hydroxide ions as part of their molecules, and reactions that don't take place in water.
Early Definitions of a Base
Chemicals were bases due to their observable properties. In this respect, bases were substances that tasted bitter, were slippery and turned litmus dye from red to blue. When you added acids to bases, both substances lost their characteristics, and you got a solid material or salt. Bases got their name from these reactions because they were the "base" chemical to which you added the acids.
Svante Arrhenius proposed a more general definition in 1887. Arrhenius was studying ions in water solutions, theorizing that table salt or NaCl dissolved in water by separating into positive sodium ions and negative chlorine ions. Based on this theory, he thought bases were substances that dissolved in water to produce negative OH- ions and positive ions. Acids on the other hand, produced positive H+ ions and other negative ions. This theory works well for many common chemicals such as lye or NaOH. Lye dissolves in water to form positive sodium Na+ ions and negative OH- ions, and it is a strong base.
The Arrhenius definition does not explain why substances such as NaCO3, which don't have a hydroxide ion that can dissolve in water, nevertheless display properties typical of bases. The definition also works only for reactions in water because it specifies that bases have to dissolve in water.
Acids and Bases in Chemistry
The Arrhenius definitions are correct in that they identify the hydroxide ion as the active component for bases. For acids, the Arrhenius definitions specify that the acid substance dissolves to form positive hydrogen H+ ions, the corresponding active component for acids.
These definitions may apply to substances outside of water solutions that don't have hydroxide or hydrogen ions. Instead, bases can be substances that when dissolved in water increase the number of hydroxide ions in the solution. Acids similarly increase the number of hydrogen ions. This broader definition successfully incorporates all substances that behave like a base into the more general category and describes what bases are in chemistry.