While the number of women receiving bachelor’s degrees in physics in the U. S. continues to increase, the number of men is increasing faster. The representation of African Americans, Native Americans, and Hispanics in physics continues to be extremely low. The American Institute of Physics (AIP) collects data on the representation of women and members of other under-represented groups in physics at all levels, from high school students to faculty members. However, the data on representation do not tell the whole story. Data from a global survey of physicists show that women do not have access to the same opportunities and resources as men do. Data from astronomers show that less than ideal relationships with advisors and the two-body problem contribute to working outside the field, and that these factors affect women more than men. These types of data are essential for designing programs that will increase the representation and retention of women and minorities in physics and astronomy.