I consider two usages of the expression “gauge theory.” On one, a gauge theory is a theory with excess structure; on the other, a gauge theory is any theory appropriately related to classical electromagnetism. I make precise one sense in which one formulation of electromagnetism, the paradigmatic gauge theory on both usages, may be understood to have excess structure and then argue that gauge theories on the second usage, including Yang-Mills theory and general relativity, do not generally have excess structure in this sense.