Six yeast strains belonging to Rhodosporidium toruloides, Lipomyces starkeyi, Rhodotorula glutinis and Cryptococcus curvatus were shake-flask cultured on xylose (initial sugar—S0 = 70 ± 10 g/L) under nitrogen-limited conditions. C. curvatus ATCC 20509 and L. starkeyi DSM 70296 were further cultured in media where process waters were partially replaced by the phenol-containing olive mill wastewaters (OMWs). In flasks with S0 ≈ 100 g/L and OMWs added yielding to initial phenolic compounds concentration (PCC0) between 0.0 g/L (blank experiment) and 2.0 g/L, C. curvatus presented maximum total dry cell weight—TDCWmax ≈ 27 g/L, in all cases. The more the PCC0 increased, the fewer lipids were produced. In OMW-enriched media with PCC0 ≈ 1.2 g/L, TDCW = 20.9 g/L containing ≈ 40% w/w of lipids was recorded. In L. starkeyi cultures, when PCC0 ≈ 2.0 g/L, TDCW ≈ 25 g/L was synthesized, whereas lipids in TDCW = 24−28% w/w, similar to the experiments without OMWs, were recorded. Non-negligible dephenolization and species-dependent decolorization of the wastewater occurred. A batch-bioreactor trial by C. curvatus only with xylose (S0 ≈ 110 g/L) was performed and TDCW = 35.1 g/L (lipids in TDCW = 44.3% w/w) was produced. Yeast total lipids were composed of oleic and palmitic and to lesser extent linoleic and stearic acids. C. curvatus lipids were mainly composed of nonpolar fractions (i.e., triacylglycerols).