Updated: January 24, 2018
Several days ago, I showed you how to install the January 2018 Meltdown patches for Windows, even if you do not have an anti-virus installed, as Microsoft decided to blacklist certain non-compatible anti-virus programs for the time being, but didn't bother explaining that this also affects people with NO anti-virus. I guess such radical departure from sheep mentality is unheard of.
Now that the patches are installed and running - I tested this on SIX systems - infinitely more than the "tech media" that simply went about repeating what Microsoft said without actually checking anything, the next thing to examine is performance and stability. Are Meltdown and Spectre patches good for you? Most importantly, do they slow down your computers?
Let's clarify a few things
Before we begin, I want to make sure you fully understand my position.
First, I do own a small number of Microsoft shares. So seemingly, I "ought" to be inclined to support the company line regarding performance - Windows 7 supposedly takes the biggest hit, and Windows 10 less, right - but this is not the case. I don't care what Microsoft says. Any benchmark result from Microsoft on Windows is meaningless. It's asking fast food vendors if their food is healthy. We need NEUTRAL, third-party results.The reason I believe Microsoft is a useful company with a decent future prospect is not because how supposedly fast Windows 10 is compared to older releases, hint: this is not the case. I've already talked about this plenty of times.
Second, I want to pull rank. I have done EXACTLY this kind of work - performance tuning, optimization and benchmarking all the way down to the lowest level in CPU architecture for the best part of my career. I even have almost a dozen patents on scheduling, power management, data center orchestration and out-of-band protocols, all related to hardware platforms. You can search online - my name + patents, and see what you get. Then also check my IEEE papers.
I have done this on huge scale, in intense HPC environments. A data center deployment with less than 10,000 servers? Boring. Memory access thingie? Old game. Issues of the Meltdown and Spectre nature and even much more complex were my fun, bread and butter, and I have written a book on problem solving that is basically the sum of my experience and knowledge of all these topics.
Just for fun, you may actually want to read these two articles - exactly how to debug and troubleshoot problems like this - we touch on system calls and timings, interrupts, context switches, CPU counters, lots of cool and nerdy pointers:
Linux system debugging super tutorial
Slow system? Perf to the rescue!
Benchmarks are useless
Yes they are. Synthetic tests are good for enterprises. They mean nothing for the home environment setup. I talked about this in regards to browsers. We touched on this recently again vis-a-vis Firefox and Chrome.
Benchmarks are useless because they do not reflect the reality. Based on raw figures, a car like BMW 330d can do 0-100 km/h in 5.3 seconds and breeze flat out at 250 km/h. But then, you go on a merry little roadtrip to the UK and its steam-era road network, and you're lucky if you even hit 100 km/h like ever. What matters is the real-life human usage.
In fact, the subject of UI performance and responsiveness is extremely complex, subjective and difficult to measure. Just imagine two people doing the same thing. Yet, they live in different parts of the world, use software differently. Add things like running programs, user's age, user's perception of speed, skills, whatever. Distraction, browser addons, Flash plugin version, active background services, graphics card drivers, hard disk cache and fragmentation, memory bus speed, Internet connection, the number of permutations is endless. Literally no two people EVER use their computers the same way.
I talked about this in one of my LinuxCon 2014 presentations - called P-Factor, which is an attempt to create a mathematical formula, based on IO metrics, that can evaluate what may be perceived as slowness for users. Some things that you may find intriguing - for example, pure CPU load (as high as 500 - theoretically 500 CPU cores) on a single eight-core system induced no lag in the UI as long as there was no I/O load.
When a certain benchmark (WhateverMark) completes in 556 ms, what does that mean? Can you translate that to how fast will the Facebook page open to someone who's on a Wireless internet connection in a coffee shop while typing with one greasy finger? Or how long it will take to send four photos of your cat to your grandma?
Importance of the patches
In general, there's a lot of noise and panic around this topic, making it sound scarier than it really is. But the issue is not as catastrophic - from end-user perspective - as it sounds. It is VERY disruptive to the industry, but much less so for individuals.All of the disclosed vulnerabilities have a local attack vector. This is IMPORTANT on multi-user systems, including shared platforms, cloud and hosting providers. This is much less important to home users, often single users (in the proverbial sense).
Local attack vector also means that if your Internet-facing programs are patched, you should be fine really. Indeed, most if not all browsers received the first round of updates that harden them against these vulnerabilities. That covers 90% of use cases already.
Now, if you freely let random executables run on your box unfettered - you have a much bigger problem than any which two vulnerabilities you want to name. This is not something that any amount of patches will fix.
The Meltdown and Spectre vulnerabilities have been around for a long time - 20 years easy, based on what I read regarding different CPU architectures. This means the issue is so complex and difficult it took two decades to discover - or someone has been abusing it quietly for a long time, and we never knew about it. Which means nothing much changes from that angle.
BTW, if you think the vulnerability age on its own is bad (the last big one was 17 years), it's not. The more we use computers, the more history we will have. So it makes sense that in 50 years, someone will find a 63-year-old vulnerability. At the moment, we don't have that much desktop and laptop use legacy, so the timeframes are shorter.
Actual tests
So what did I install and run?
- In Windows, I only installed Windows patches - no firmware updates.
- In Linux, I installed whatever comes through the repos - kernel and firmware.
Why no firmware on Windows, you may ask? In some cases, there aren't any available. But in general, for single-user systems, I feel there's no need, as I've just explained above.
Now, let's see what results I got.
Case 1: 2010 laptop
An old HP machine. First-gen i5 processor, Windows 7 SP1, Nvidia graphics, limited user, EMET and SuRun for security, and no anti-virus whatever. In fact, this is the important factor in all of my cases. No real-time anti-malware thingie using CPU cycles.
Everything worked just fine after the updates. No perceived slowness, and I am EXTREMELY sensitive to even tiniest changes, in well anything. I once spotted a 0.6-deg camber change in the rear suspension of a car, and had to take it for realignment. I tested full HD video in a browser, did a few things in parallel. Smooth as a peach. I can see no significant difference whatsoever compared to before the updates.
Case 2: 2011 desktop
This is a gaming rig from 2011. Extremely capable still. Windows 7 SP1, admin user, EMET for security, 16 GB of RAM, i5 processor, five WD Caviar Black hard disks, Nvidia GTX 960 card - upgraded a few years back. Everything works silky smooth. I tested a bunch of computer games, too. I see no increase in CPU or GPU utilization under heavy load. No increase in context switches. No lag at all. Apps opens as quickly as ever before. I also tried HD media streaming, image manipulation in GIMP, TrueCrypt container use, you name it. Splendid.
Case 3: 2014 laptop
Here we have a Lenovo IdeaPad Y50-70. My most powerful mobile device - it's also got 16 GB of RAM, Intel HD plus Nvidia GTX 860M, this one being equivalent to a desktop GTX 580 card - about 10-20% less than the GTX 960 card, but then it also has a newer i7 processor, and with HT on, this means eight threads rather than four threads for the gaming desktop. The system runs Windows 8.1, admin user, EMET for security.
I even feel it's FASTER now than it was before the updates. But then it also has the latest Nvidia drivers, which could also be beneficially contributing to the end result. While playing ArmA 3 and Cities Skylines, everything was hunky dory. For instance, Cities Skylines tolled about only 30% GPU and 45% CPU with a 100K map and tons of mods = same as before.
Case 4: 2015 laptop
My multi-purpose test rig - Lenovo IdeaPad G50-70. It's got 8 GB of RAM, an i3 processor, Intel HD graphics. Windows 10, limited user, Exploit Protection for security. After the updates, everything worked beautifully.
I even tried streaming TWO 1080p videos at the same time, one in VLC and one in Chrome, and again, there was no lag, no problems of any kind. Switching between apps is fast. Everything works. No errors, no complains. If there is any degradation, it's below my threshold of sensitivity as a human and a nerd.
Other cases
There are still more - a 2012 desktop - similar to the gaming rig, with a slightly less powerful processor and graphics, but again, no issues. We also have a 2013 VivoBook, and this one dual-boots Windows 8.1 and Ubuntu Trusty. On the Windows side, we have the familiar security setup - admin plus EMET. AGAIN, no slowdowns.
No vendor updates, though.
Bonus case: Various Linux distributions
I even tested half a dozen different systems, on the Lenovo G50, the HP and the Asus VivoBook above, which is probably the most interesting one, as it's also a production system. In fact, I never had Intel microcode installed in Ubuntu 14.04 on the ultrabook, and I made sure to install it after upgrading the kernel (with Meltdown patches).
Similar to the 2014 IdeaPad laptop, things feel FASTER after the updates. Really snappy. Refreshingly delightful and trustworthy, as Trusty always was. Everything works fast and true and without any errors. Before and after the patches, just the usual fun workload:
The fluctuation of system numbers is irrelevant - the actual load goes up anywhere between 40 and 90% CPU, and the rest of the metrics accordingly. Whatever the change is, it's not noticeable.
There's no perceivable slowness of any kind. So that further helps our experiment, as we have a completely different set of operating systems and kernels to confirm the Windows findings.
Extras: Firefox and Chrome mitigations
I also tested Firefox 57.0.4 and Chrome 63 with the new site isolation flag. Again, if there are any differences, they are well hidden from human perception in every sense of the word. Yes, I did try Chrome 63 with the site process isolation:
chrome://flags/#enable-site-per-process
What I like about this whole drama
So I have to say, I am quite pleased with how hardware and software vendors have been handling it. Spot on. No crashes. No bugs. No performance impact. Very nice.
What I dislike about this whole drama
It's the whole passive-aggressive language of the "tech" media. Everything is phrased lawyer-like so no one is liable in any way. Tiptoeing around the problem by regurgitating bullshit. Not only is it annoying, it's also a waste of time and space. If the whole purpose of articles is to copy-paste officials statements from Intel or Microsoft, you might as well as not bother writing anything.
But then, speaking of Microsoft, statements like "expect some users will notice a decrease in system performance ..." are pointless. What does that even mean? People should also expect to die one day, it does not mean it's an issue right now. Perception is subjective. Some people won't even notice if you turn their computer off.
The mention of font rendering is another one. As I've shown you in my linked article above, and there we debugged EXACTLY that, a font problem, and in the tutorial on how to setup Meltdown patches (and the fact WU only reads registry on startup), this is a bad argument. Most if not all applications will cache fonts only during the loading time. Memory operations are fast. GPUs are really fast.
But wait. Rendering? Uh. So, this is interesting. Are we talking mathematical operations, CPU processing, GPU processing, what? Now, rasterization, anti-aliasing, hinting, screen resolution, display refresh, what are we talking about exactly? Here's a nice article on the topic, with some numbers.
I know why Microsoft - and other companies - must disclaim their findings, and also err on the side of caution (pessimism), but if that's the case, don't write anything. There's no value in implied statements that Windows 10 performs better than Windows 7, because that just erodes credibility. Let me explain why.
Take a computer from 2010 - at that time, Windows 7 was the most current Microsoft operating system. Provided you have the best set of drivers for your hardware, you have an operating system that performs at the peak of its ability.
Now, install Windows 10 on it. You will not get some super-magical improvements. Perhaps, yes, a kernel function called _NT_win10 may be faster than _NT_win7 in certain cases. But to claim these huge performance improvements means: 1) monumental kernel changes in Windows 10 compared to Windows 7, which does not make sense given the fanatical backward compatibility that Microsoft has for its software and the incremental nature of progress in software 2) this implies that Windows 7 is a sub-optimal system and that Microsoft kernel engineers did not know how to make the best of the available hardware back in 2010. That's nonsense.
New hardware, YES, I AGREE. If you now install Windows 7 on a new hardware platform with a brand-new CPU that was designed AFTER Windows 7 stopped receiving kernel upgrades, it is possible that the older system does not know how to make best use of the new features in hardware (say Thunderbolt, USB 3.0, SSD, CPU extensions). In this case, using the latest, most modern Windows makes absolute sense. I have already argued that when I switched from Windows XP to Windows 7. Make operating system changes to match hardware. In-vivo replacements are meaningless. Upgrading one version of the operating system to the next will not render miracles.
I have had a chance to verify this statement in the enterprise world, too, with commercial flavors of the Linux operating system. Newer versions and kernels rarely gave significant improvements (or even meaningful ones) on the existing platforms. They always performed better on brand new hardware. It was never the just kernel itself that made the difference but the combination of kernel, glibc, compilers, software versions, and hardware drivers.
Given all this, even if there's no hidden agenda, people will see one. Then, what do you expect from Microsoft? To write on their official blog: No, don't use Windows 10, we don't need the money?Which is why you ignore any consumer advice from the product owner, because it is subjective. Even if 100% true, it's still subjective. And this is where we meet the failure of the "tech" media head on.
Hundreds of websites hurried to talk about this topic - literally, verbatim copy-paste of Microsoft and Intel statements. Hey, anyone savvy enough to use Ctrl + C, Ctrl + V can do that. And then, the focus stays on drama and lament. No one bothers to check the facts, test the statements, present data (a few brave souls have started showing some actual numbers, but it's still mostly speculation like the nature of these vulnerabilities).
I believe people should only talk about what they know and understand. In my mind, when someone claims to be a techie, that means they have deep knowledge of what they're writing. If I wanted empty pre-sales and sales pitches full of fancy buzzwords and complicated terms (like branches and memory management), I'd go to a life-guru session so I can have my first-world existence given a new meaning.
Actually, I feel sad for all these people who go to work in the morning, sit down, hash out a few hastily written articles, and call it a day. Where's the passion? Where's the desire to help people? To tell someone Meltdown will slow their machine but not give them an alternative, or even a suggestion on what to do next is just doomsday preaching. Especially since the reality isn't bad at all.
So from a tech perspective, what do we have here?
So I tested pretty much all possible scenarios - all with Intel processors mind, as that is what I have. I do not currently own an AMD system. Anyway, Windows 7/8/10 all work as fast as before, if not better, with no discernible impact from the patches. The platform is irrelevant. 2010 through 2015, Asus, Lenovo, HP, it's all good. As it happens, I also had a chance to test five different generations of i3/5/7 processors, Nvidia and Intel HD graphics as well hybrid, limited and admin user, and still other mixes.
I tried gaming, including some pretty decent titles. I tried media streaming locally and online, browsing, image manipulation. Word processing, encryption software, and so on. Not a single error, crash, lag, delay, or anything. I can only hope all future updates are as nice as this batch.
One thing that comes to mind - I use no AV software. I see no point in that. I prefer using things like limited user, EMET and/or Exploit Protection. Microsoft has these phenomenal security frameworks, and they beat all and any classic, blacklist-approach tool. And given the QualityCompat registry hack need to install the Meltdown patches, I presume the slowdowns are indeed possible if people use anti-virus programs, as there's a lot of focus there. But I can't test that, and that's not something I want to do even if I could. The whole teach a man how to fish kind of thing. I prefer to help and educate people on the philosophy of problem solving and critical thinking.
Finally, don't do what everyone does
This applies to everything in life. If you do like everyone else, you will be like everyone else, and most people are just mediocre, working their entire lives to pay off loans and mortgages. You don't want that.
If they tell you to buy, sell. If the world is clamoring security, panic - you relax. In this case, a bad moment in the IT industry is not the right time to make software or hardware upgrades. Do not be pushed into buying new computers, to switch vendors, upgrade operating systems, or anything like that. If you planned pampering yourself with a new laptop or a desktop, go ahead, sure. But reacting to this pandemonium with your wallet is exactly the wrong thing to do.
The best you can - and should do - is not trust anyone out there, not even myself. You should read this article, and then run your own independent tests that suit your skill and needs. Make sure that it works for you. I am highly confident you will see virtually no deterioration in your system performance - of course IF you run setups like mine, i.e. no real-time anti-virus software. If you do use them, this article is probably not the best piece of advice for you.
Conclusion
If you ask me for numbers, I am not going to provide them. Because they don't mean anything. UI slowness is subjective. Benchmarks are important - for the industry. If you run a 400TB database cluster over 55 nodes interconnected over 10Gbps fiber, yes, perhaps you need every cycle you can get. Home environment? Forget it.
That said, from my subjective, personal and still exhaustive testing on a variety of hardware platforms, with a mix of operating systems, scenarios, conditions and programs, I see no perceivable performance degradation of any kind, be it a new or old system, or anything. Everything works well. Everything is stable. All I can say, great job vendors. As for the media out there, well, watch out for those clicks. Sensationalism sells.
Cheers.