System76 is one such company which offers Ubuntu only PCs. The company has launched two new laptops pre-loaded with Ubuntu 12.04 -- 14.1" Lemur Ultra and 15.6" Pangolin Performance.
Both PCs are available with the choice of Intel Core i5-2450M Processor (2.50GHz) or Intel Core i7-3610QM Processor (2.30GHz). These laptops come with 4GB RAM, but you can upgrade it to up to 16GB for additional cost. These laptops sport Intel GPUs.
Why Ubuntu Pre-loaded PCs?
The advantage of Ubuntu (or Linux pre-installed) PCs is you can be rest assured that the hardware will work out of the box and you won't have to struggle with drivers. You can also expect some level of support for Linux with these PCs. However, the fact is most modern PCs will work out of the box with Linux (as Linux kernel developers, especially Greg KH have worked hard with vendors to ensure their drivers are available for Linux).
Still at times a bad hardware can give you nighmares so its better to buy a Linux supported PC.