Primary prostate cancer almost always has a luminal phenotype. However, little is known about the stem/progenitor properties of transformed cells within tumors. Using the aggressive Pten/Tp53-null mouse model of prostate cancer, we show that two classes of luminal progenitors exist within a tumor. Not only did tumors contain previously described multipotent progenitors, but also a major population of committed luminal progenitors. Luminal cells, sorted directly from tumors or grown as organoids, initiated tumors of adenocarcinoma or multilineage histological phenotypes, which is consistent with luminal and multipotent differentiation potentials, respectively. Moreover, using organoids we show that the ability of luminal-committed progenitors to self-renew is a tumor-specific property, absent in benign luminal cells. Finally, a significant fraction of luminal progenitors survived in vivo castration. In all, these data reveal two luminal tumor populations with different stem/progenitor cell capacities, providing insight into prostate cancer cells that initiate tumors and can influence treatment response.