An important question we need to ask is what is the purpose of the reviewer? Most people watch or read reviews to find out if something is worthwhile to purchase and to see how it compares to other in that price range. An important aspect of reviewers is to help their audience gain context about the current products they are reviewing, the available competition and that category in general. It is also, important to try to understand your audience that your targeting. 

A review that upon first reading irked me upon further reading is actually a good review not flawless, but good. In the BuzzFeed’s review of the M1 iMac John Paczkowski talked about how the original iMac was so much better than his 1995 Compaq Presario. This seem like a strange reference to mention a nearly 3 decade only computer, but it gives the reader the context of why people cared about the original iMacs. He then goes on to mention his current computer a 2015 iMac and how it is much slower than the M1 and cannot have as many applications open. He then proceeds to say wirecutters top pick HP All-in-One 24-df1036xt is worse because it has worse performance, a worse screen and is not worthwhile even with its lower price. My complaint is that the reasoning behinds these opinions is not well supported. This review does not provide benchmarks or a quantitative Data besides he could have more tabs open. However, the reader is given actionable information that will help them make a purchase decision. I am now going to show you how reviewers can go wrong with quantitative numbers and be lead astray by numbers and specifications. I will be comparing the M1 Ultra Mac Studio with the Alienware R13 the media coverage.

(Many) Reviewers are Basically Useless

Both Youtube channels Linus Tech Tips and Gamer’s Nexus have done reviews of prebuilt computers. I am going to focus on Gamer’s Nexus Coverage of the Alienware R13 and compare it to other outlets. I am picking this computer because it is the flagship device from a major PC manufacturer. It also cost around the same as Apple’s Mac Studio another flagship device. We can see how these computers compare in benchmarks as well. 

Gamer’s Nexus released 4 Videos about the computer and its components

1. Crazy Bad $5000 Alienware Gaming PC: R13 Aurora Tear-Down (April 24 ,2022)
2. The Worst Pre-Built We’ve Ever Reviewed: Alienware R13 $5000 gaming PC Benchmarks (May 1, 2022)
3. “Fixing” the Alienware R13 Dumpster Fire (May 22, 2022)
4. Sometimes Dell Actually Tries: Dell RTX 3090 Review, Tear-Down & Benchmarks (July 2021)

A quick summary of the problem that Gamer’s Nexus found was that the Alienware ran so hot and was so power limited that its CPU the i9-12900KF performed like an i7-12700K. A power limit means that the system did not provide enough power to the components it was due to software in Alienware’s case, but can also be a power supply issues as well.

Looking at Amazon on July4 , 2022 the i9 cost $549.97 and the i7 cost $377.98 (USD). That means that in the a flagship computer your CPU cannot perform to its full potential and performs like a CPU that cost more than $150 less than the one inside the computer. Also, the computer uses a R13 specific case, motherboard & power supply so you cannot reuse any of the components for a new build. Removing the front panel caused the internal temperature to drop by 10 degrees and during load the CPU was nearing the dangerous maximum temperature. The fans are loud. All the ram is green on an all black Motherboard & Graphics card PCB it does not match the aesthetic. The RAM that Alienware put inside the computer is Slow DDR5. The computer does not have a thunderbolt ports. The R13 is not meaningfully upgradeable as you are thermally and form factor limited. The front inputs are on the motherboard instead of a separate board. If your front inputs get damaged it requires you to replace your entire Motherboard not just the input board.

This is not the case with custom built computers and this is not the case with M1 Ultra Mac Studio as in seen in the Max Tech Teardown . Apple has more replaceable components so that if any port gets damaged you replace the breakout board not the whole motherboard. However, Apple locks it components to CPUs so enjoy trying to fix it but it can be physically be fixed. It also does not have 10gigabit which is used for High end connected storage and is standard on the Mac Studio. My custom built computer which is half the price of this computer has 10gigabit ethernet and thunderbolt so it is frankly unacceptable. When we visit the website OpenBenchmarking.org the i9-12900K (A variant that has same boost and clock but more enterprise features) it is in the 75th percentile of a site that allows dual core servers workstations. The i9-12900K in many benchmarks in in the 99%+ percentile so saying it is faster than any other system is a given. The real question is what value does Alienware bring to this components! 

Other Media Coverage

Many of Many (An Australian Men’s LifeStyle site)

They mention the loud fans but are not as annoyed as Gamer Nexus. They never really compared it to other systems on the market or talked about thermal throttling. They never compared it to an external test bench. They did mention that boutique builds would use nicer material than was used on the Alienware. They also did not compare it to other offering.

Tech Targeted Media

PC Mag

When scrolling across the page they show similarly priced system but the prices are not as configured for the review but as the base system price which is misleading. They also mention that it does not feature glass side panel on the case that high end competitors have, but plastic. This computer cost $4.5k+ so maybe Alienware could put some nice glass on it I mean it is only a couple of months of rent in a two bedroom apartment in NYC (https://www.apartmentlist.com/renter-life/cost-of-living-in-new-york). They do not mention that it does not have comparable connectivity to other products in this price range and that the device thermal throttles. The Alienware system also performed below the test bench they used in the review for the i9-12900K they reviewed standalone which was not mentioned. System to system variance is real, but for $4.5k you can tune a system. Compared it to other offering and it was worse with a more powerful GPU. Also, the benchmarks never ran for a longtime so the thermal system was never really taxed. So, more benchmark and no real additional useful information.

Hot Hardware

Mentions fan noise of previous generations. They mention the wifi chip could be better I mean it is only $4.5K+ who needs better wifi. They did not experience any thermal throttling in testing and device even got the Editor’s Choice award even when it did not perform as well as on some of the benchmarks for the standalone i9-12900K that Hot Hardware reviewed. They also did not use almost any of the same benchmarks for the standalone beside Speedometer so it is very hard to show a drop in performance. 

If you look at these website you would think that I am making up this thermal throttling issue, but I picked this device because I know it thermal throttles. So, the YouTuber JustIn Tech Tips made a video called “How to Mine Crypto on the Alienware Aurora R13 – Without Crashing!” he had to dissemble the GPU and apply new thermal paste on a New $4.5k+ computer. The GPU temperature dropped by almost 20 degrees Fahrenheit and allowed him to Crypto mine for 24 hours a day instead of instantly crashing! I am not a fan of crypto and recommend Nicholas Weaver lecture talking about its issues. JustIN Tech Tips also replaced the stock fans with a pair of after market fans (Noctua NF-A12x25) he found that it was quieter even at maximum speed. I am avoiding talking about his testing methodology because that is a whole another article and most channels do it wrong. He did see that when the where fans not at full blast they had worse temperatures. The difference in fan performance could be do to a poorly optimized fan curve or how quick it makes the fan spin for a given load. Remember Alienware only contribution to this system was the power supply, cooling, circuitboard and chassis design. The fact that both JustIn Tech Tips and Gamers Nexus could improve the system means that Alienware’s engineer where not trying very hard. 

Now let’s see how the Alienware compares to the Mac Studio with M1 Ultra. First UFD Tech found that the Mac studio was very efficient and could mine and does not mention the device crashing when trying to mine on the device or the standalone PC he built. In the PC Mag review they mention even under load the Mac Studio fans do not get loud unlike the Alienware. They mention that the Mac studio is not user upgradeable ,but do not talk about how the Dell used not standard parts. In Linus Tech Tips review of the Mac Studio even with using unrealistic workloads they could not get the mac studio to thermal throttle and the fans never spun up past 50% of fan speed.

 The Tech Journalist Pipeline

I could only find a couple, but Glassdoor says it all a Journalist Total pay is $85,580 ($52,551 Base Pay) and for an Electrical Engineer $102,303($81,649 Base Pay) so becoming a journalist will cause you to loses $15k+ in salary on Average. Making it hard to recruit any one from the industry with knowledge. PC Mag’s Glassdoor shows that a staff writer makes somewhere between $76K – $81K Base Pay. This is actually industry competitive base salary, but it does not show the difference in working environment. 

A big incident of the poor culture in tech journalism is the infamous Verge PC Build which showed the incompetence of need for quick video for clicks by releasing sloppy videos. Check out Timmy Joe PC Tech video on the subject for a full context. Eric Jackson a former Contributor for Forbes wrote that “Why is there so little criticism of tech companies by mainstream business journalists today?”. Where he lamented that few journalist add their own opinion because they fear being wrong or losing access to companies. He points out the mainstream Business Press is not writing Long Form stories covering tech companies. Jackson says “Sure, you’ll get the occasionally bright blogger shining a spotlight on truth but the mainstream business press really needs to start pulling its weight.” YouTuber are the bright bloggers of the modern era.

Tech journalism has become tedious product journalism where printing the spec sheets for mass produced consumer products is celebrated as a great story and where there appears to be little understanding of bigger picture stories about how our digital technologies are transforming our industries, cities, and our societies, at a pace and scale that’s never been seen in our history.

Tom Foremski article in “The role of tech journalism in a post-technology world”

His words are still applicable today as very little context is given even in bench mark heavy reviews. For instance the technology outlets who benchmarked the Alienware R13 got burned because Alienware made sure that short benchmarks would run fine on the machine. As someone who works in a companies lab we do know what test your most likely going to run and we will see how we compare to thecompetition before hand.

Closing: Why does this matter?

The average consumer does not have the ability to do high level testing or try a large set of products that are available so they need journalist to provide them with context. The technology companies have basically spent over a decade with minimal journalistic oversight and it shows. Companies are savvy and know what is being tested and reviewers must update methodology to make sure the readers understand the benefits and downside of products.