Like "why is the sky blue?", this is a pretty complex topic! The short answer is that not all papers do, and it depends entirely on what fibers the paper is made of, the processing and purification steps it has gone through, and the additives or residues that are left in it.

Asian-style papers are made from the core of tall grasses (bast fibers) and Western-style papers are made from 100% undyed cotton and linen rags or cast-off fibers from spinning. After being picked or washed clean of impurities, the long fibers that are left are pure cellulose, which is actually colorless, but reflects light opaquely and we see the color white. Newsprint and some other papers are made from ground wood or straw, also cellulose, but which have more non- or hemi-cellulosic compounds by weight, including lignin. Lignin and other non-cellulose compounds are subject to oxidation - and when they pick up extra oxygen it alters their molecular structure (becoming chromophores) in a way that absorbs and reflects wavelengths of light differently - and changing the colors we perceive with our eyes to yellow and brown in visible light. Pure cellulose papers do not oxidize as much as impure ones, but they do tend to become warmer in tone than when first made. This may be because of additives (sizing, fillers or coatings) aging, impurities in the water they were made in, or picking up and adsorbing pollution that affects the color of the cellulose. For more information see this research abstract and the series of articles. For a more general survey of how paper is made, see this great resource from the Robert C. Williams Paper Museum.

As for photographs, fading and darkening are two different phenomena, and we could write a topic on this in particular, but many of our colleagues have already done a great job of it. Check out the National Archives' discussion on fading in color photographs and the Image Permanence Institute's types of photograph deterioration.

We can chalk this up, in general, to entropy I guess.

New research has identified a culprit, a specific bond in cellulose itself that is more likely to yellow. Beyond the questions the article raises for naturally aged historic works, this raises philosophical questions for modern contemporary art. How much do we appreciate yellowing as a sign of age? Do we pre-treat every contemporary artwork to prevent yellowing? Will paper manufacturers embed more antioxidants into traditional artists' papers?