Cryogenic fracturing has been explored in recent years as a waterless fracturing method for well stimulation to avoid issues encountered in water-based hydraulic fracturing. Cryogenic stimulation using liquid nitrogen applies large thermal gradients on reservoir rocks to induce fractures. This study investigates the initiation and proliferation of cryogenic fractures from boreholes under external stress on specimens. We flowed liquid nitrogen through boreholes drilled through the center of transparent PMMA cylinders under uniaxial stress and monitored fracture proliferation, temperatures, and borehole pressures. Our results show that the effect of stress resembles that of hydraulic fractures such that fractures propagate more in the direction of the stress. Under loading perpendicular to the borehole axis, a cloud of annular and longitudinal fractures extends more in the direction of loading. Under loading parallel to the borehole axis, longitudinal fractures dominate, and annular fractures become more suppressed and more sparsely distributed than those of unconfined specimens. Even if fractures are driven to initiate against the influence of stress, such as those from a boundary edge of a high stress concentration, they gradually deflect in the direction of stress, similar to hydraulic fractures from perforation holes that curve toward a direction perpendicular to the minimum stress direction.