more advanced Lab tests for computers

Faced with the rapid development of technologies, the creation of new varieties and the interest of enthusiasts, Labo Fnac is changing the protocol of computer performance evaluation. Even more demanding and closer to the reality of use, they allow you to test all products, from Cloud Computing (ChromeBook and Windows S) to central units dedicated to Gamers, including all-in-ones. One Protocol to rule them all!

Indeed, it was built on games like the old protocol Call of duty modern warfare. It’s a demanding title, but it’s been the same since 2007. If it represented the highest at the time, it is limited to 91 fps (pictures per second) and is no longer suitable for testing the latest generations of graphics cards. The same goes for other popular titles ugly rally and cry awayeven if the measurements taken are not included in the calculation of the score.

On the office side, Adobe solutions were used, as well as important Word and Excel programs. The tests consisted of creating large files with various filters and macros, documents containing hundreds of chapters or spreadsheets.

However, this software has evolved into so-called “cloud” solutions, whose performance is now more dependent on the quality of the Internet connection and available servers, making all measurements inconsistent with the associated rating system.

So what to do? Labo Fnac chose to start again with the same care to be fair among all the products tested.

Internal tests

Advances in technology have prompted the laboratory to develop testing protocols as well. But even if that means changing everything, you can do it the other way around. Javare Traore (computer test manager), instead of using classic test programs like most specialized sites, preferred well-known open source programs such as ImageJ, developed by the US Institute of Health and widely used by scientific research. community for image processing. Of course, protocols are designed to estimate the use of computers by users, and ImageJ will be used to evaluate performance in office automation.

By the way, just install the built-in app called PerfLab to perform some measurements right out of the box. It should be noted that the autonomy test has also evolved somewhat and there are still measurements at the level of the sound part and screens related to laptops.

Pushing computers to their limits

The new test protocols have a clear goal: the machine must be used to its maximum capacity, while targeting each component individually, for example, all cores of the CPU (main processor). Something that cannot be done with Word, for example.

For this first test, Labo Fnac uses the popular Blender software. A reference in the world of 3D modeling, which allows you to use new graphics card functions such as Ray tracing at up to 360 frames per second in HD, will be the basis for game recording.

The protocol consists of 3D modeling of the entire Earth, the Moon and the Milky Way as a background. And as if that wasn’t enough to use a variety of textures up to 38K (21,600 x 10,800 pixels) developed by NASA.

As we wrote above, this test allows you to specifically target the CPU by performing three different renderings at 2K, 4K and 8K. In the screenshot below, we can see that all cores are required when rendering is enabled. The video card is also used, and the operation may take several minutes depending on the performance of the computer. That’s also one of the advantages of this particular test: it’s not about being satisfied with simple top speed.

After the test is completed, a file is created and can no longer be changed, and it is the file where the results are written. Also note that the operation is repeated at 2, 4 and 8K not only for the CPU, but also for the GPU (graphics processor).

But that’s not all. The tester also uses a BMW 27 to confirm these homemade 3D renderings. This is a rendering specification file widely used on the Internet. The latter allows us to compare rendering performance with a global database.

Finally, there is also an animation presentation with Eevee, Blender’s real-time engine based on the OpenGL cross-platform interface. The latter allows you to target the zero GPU, that is, the main graphics chip, which is generally the least powerful and consumes the least. During this step, animation rendering is performed on 11 images. Then it remains to note the duration of the rendering of the latter.

All the results of these tests are imported into the laboratory database. Incidentally, the lab also observes write times on internal storage, be it a good old hard drive or SSD. Similarly, the log file collects other useful information such as RAM performance.

Unable to test Nvidia graphics card like AMD model or M1 processor. The new Fnac Lab protocol allows you to use the rendering engine most suitable for the graphics component (directX, CUDA, OptiX, Metal) and adapt to all platforms MacOS, Windows, Linux, ChromeOS, Windows S. Proof of demand from these tests, even the extremely powerful RTX 3090 he bent his knees during the assessment.

Tests adapted to real use

In addition to raw performances, interest is also focused on the specific uses of users. In the next step, the protocol simulates the needs of a photographer or graphic designer who will handle large files.

This batch processing uses photos to be processed by ImageJ software. The principle is quite simple, a target folder that groups the input photos and an empty output folder that groups the files to which the macro will be applied. This will add multiple tags to each image, resize them, and save them under a new name.

Web apps are now essential on our computers, smartphones, tablets and even TVs. To evaluate their performance, the lab uses the main video card (not the second, more powerful GPU, the GPU 0 we talked about above) and the Pixel Fill Rate test, which targets the ability to generate a number of pixels for each pixel. the second.

This test allows you to get maximum FPS and determine intelligent performance management without automatically switching to the most powerful GPU 1 when the system is under heavy load. The advantage of this script is that it can adapt to the performance of the product it is running on (PC, mobile device, etc.) and can go up to GPU saturation. The resulting curve shows the time in milliseconds on the ordinate, and the number of megapixels on the abscissa. Its shape indicates whether the component is stable or not.

After all, it is impossible to test a computer without showing interest in the gaming part. The new test protocol is divided into two parts. On the one hand, the GPU rendering seen above, and on the other, real-time rendering. The latter is again made with Blender, but also with FPS Monitor.

The program is configured to use wire rendering (Wire) first, then solid rendering, and finally rendering with the EeVee engine. raytracing, that is, to display the reverse of the animation on the screen. FPS Monitor provides the number of frames per second (fps) during three tests. These results, related to GPU rendering performance, create an overall score for the game portion.

Extensive testing in all areas

As before, other tests are carried out to evaluate screens, sound quality or even autonomy related to laptops. So, the autonomy test also improved a bit by implementing a solution for continuous video streaming from VLC.

Screens are tested with professional equipment used for televisions. Recall that the following are evaluated: colorimetry (but only in sRGB, unlike TVs tested in DCI-P3 and rec2020), gamma and orientation. Another key difference from testing on TVs is that the contrast ratio measurement is not based on the ratio of black to white levels, but rather with 5% black (to realistically compare LCD panels and LCD panels). OLED panels!). Finally, the pixel density is displayed, something we don’t do with TVs.

Finally, the audio tests have also improved a bit. Indeed, the jack plug test was removed because it was time-consuming for little interest (with the arrival of bluetooth headphones and the removal of jack ports on new computer chassis). In addition, the loudspeakers are tested in an anechoic chamber by placing the computer on a test table at a height of 40 cm. The microphone is positioned in front of the user to simulate the position of the user when sitting in a chair and in front of a laptop on a table. The screen angle is always the same to compare the results.

During this step, the frequency response measurement is usually performed with a bandwidth ranging from 63 Hz to 16 kHz (notebooks are limited in terms of low frequencies). Finally, a distortion measurement at 1 kHz for 3% distortion with the most common frequency and always the same configuration at the microphone level.

All these tests allow you to get different ratings of the radar to compare computers and accompany you in your choice.