Vista vs XP performance: Some informal tests

Who's the daddy?

After posting about the inadequacy of a recent test report I thought it would be interesting to conduct my own informal tests of Vista vs XP performance. I do not run a computer laboratory, but I guess my tests have the benefit of being real-world.

I tested several conditions on three computers. On two of them I was able to test XP 32-bit vs Vista 32-bit. I tried various combinations of Aero on or off, visual effects on or off, and UAC (User Account Control) on or off. I also tried setting Vista to run only basic services, using Msconfig.

The test suite I used was Performance Test 6.1. You can use this free for 30 days, so anyone can repeat the tests.

Caveat: perfect tests are hard to achieve. A big problem with Vista is that it has all sorts of background services that are meant to maintain your system, perform backups, check for updates, and so on. Further, if you have third-party software installed (and who doesn’t?) then it may also be running background tasks. The only compensation I applied was to wait a few minutes after start-up for disk activity to pretty much cease. I also switched off anti-virus software, and closed the Vista sidebar.

Here’s a few highlights. First, XP was fastest on the two dual-boot machines, and that was without any effort optimizing XP for performance. In both cases, the best Vista performance was about 15.5 per cent slower than XP, based on the PassMark performance index. On one machine the worst Vista performance was 28 per cent slower, and on the other 23 per cent slower.

Graphics 2D - Ouch!
The most interesting result was the poor performance of the Graphics 2D tests on Vista. A series of PassMark tests cover drawing lines, rectangles, shapes, font rendering, and common GUI operations like scrolling listboxes, moving windows or filling progress bars. On both dual-boot machines, the best Vista performance was an amazing 70 per cent slower than XP. Put another way, XP was three times faster. Since these are common operations, that is worrying.

That said, there are some specific reasons for this poor performance. See here for example:

The Windows Vista graphics system is designed to support a broad range of hardware and usage scenarios to enable new technology while continuing to support existing systems. Existing graphics interfaces, such as GDI, GDI+, and older versions of Direct3D, continue to work on Windows Vista, but are internally remapped where possible. This means that the majority of existing Windows applications will continue to work.
Hmm, re-mapping sounds slow. And here:

GDI primitives like LineTo and Rectangle are now rendered in software rather than video hardware, which greatly simplifies the display drivers. We don’t think this will impact many real-world applications (usually when a GDI application is render bound its because it’s doing something like gradients that was never hardware accelerated), but if you do see problems please let us know.
See also Greg Schechter’s notes here on GDI-rendered windows.

In a nutshell, Vista is optimized for DirectX rendering at the expense of GDI; yet, as Schecter notes:

Today and for the near future, most applications use and will continue to use GDI to render their content.
This I suspect is a large part of the reason why some tests report that XP is twice as fast as Vista. Automating Office could well hit this slow GDI issue. Note that it is not all 2D graphics that are slow - CustomPC got less alarming results for its 2D graphics tests, which by the looks of it are not GDI-based.

Schechter does not cover the scenario where the DWM (Desktop Window Manager) is turned off, but it looks as if some of the factors which make GDI slow still apply.

What’s most effective at speeding up Vista?
I did some experiments where I compared Vista Basic with Vista Aero, or looked at what happened when UAC is switched off. I got inconsistent results. On an older machine, I found that disabling Aero made a significant difference, maybe an eight per cent speed-up. On another, more recent machine, Aero was actually faster. So much changes when you use DWM that I guess this is to be expected.

UAC? Not a huge difference in these tests - around two per cent.

The biggest influence, on the basis of my imperfect testing, came when using Msconfig to switch off all but basic services and start-up programs. This speeds performance by around 10 per cent overall.

In most other tests there were modest differences between Vista and XP. This includes 3D graphics, where Vista actually scored higher than XP on one machine, and CPU, where on one machine there was less than 2.5 per cent difference between best and worst. Vista does come out significantly slower on the PassMark memory test suite, from just over eight per cent worse to over 20 per cent worse.

Conclusion? First, my informal tests suggest that XP is faster than Vista, but not normally twice as fast. Second, an application written to use DirectX rather than GDI should perform better, other things being equal. WPF (Windows Presentation Foundation) uses DirectX, but unfortunately has its own performance overhead.

If this analysis is right, then Vista is at its worst when rendering the GUI in traditional Windows applications. That will make them feel less snappy, but would not impact the non-visual aspect, like say recalculating a spreadsheet.

Then again, I only tried one test suite, so please don’t take the above too seriously.

I’d be interested in further informed comments.

This article originally appeared in ITWriting.

Copyright (c) 2007, ITWriting.

A freelance journalist since 1992, Tim Anderson specializes in programming and internet development topics. He has columns in Personal Computer World and IT Week, and also contributes regularly to The Register. He writes from time to time for other periodicals including Developer Network Journal Online, and Hardcopy.

BEA portal product springs a leak

Organizations using a popular portal server made by BEA Systems may be interested to learn that researchers have figured out a simple way for unauthenticated users to obtain every user name stored on their systems.

The user name leak resides in an advanced search function in the BEA Plumtree Portal 6.0, according to this advisory from researchers at ProCheckUp, a company that provides penetration testing services. The results included both regular user names as well as those belonging to administrators.

"What we found is that by tweaking the parameters of the search functionality, it is possible to obtain all the usernames of the target corporate portal," the researchers wrote in a report. "What makes this vulnerability attractive is that the attacker doesn't need to be logged in in order to obtain the list of usernames."

The enumeration made possible by the vulnerability is of the "dumpable" type, meaning there is no need to run a dictionary attack to find valid usernames, as is often the case with attacks on user databases.

The vulnerability has been fixed in the AquaLogic Interaction 6.1 MP1. Users not ready to upgrade can also work around the bug by making configuration changes to the product. BEA representatives were not immediately available for comment.

ProCheckUp also disclosed two other vulnerabilities affecting Plumtree that are available here and here. The researchers who discovered the bugs are Adrian Pastor, a member of GNUCitizen, and Jan Fry. ®

So many paths to Nirvana

We've groused repeatedly about the gaps in the software development lifecycle, or more specifically, that communication and coordination have been haphazard at best when it comes to developing software.

Aside from the usual excuses of budgets, time schedules, or politics, the crux of the problem is not only the crevice that divides software development from the business, but the numerous functional silos that divide the software development organization itself.

Software developers have typically looked down at QA specialists as failed or would-be developers; software engineers look down on developers as journeyman at best, cowboys at worst; while enterprise architects wonder why nobody wants to speak to them.

Not only do you have functional silos and jealousies, but the kinds of metadata, artifacts, and rhythms vary all over the map as you proceed to different stages of the software lifecycle. Architecture deals with relatively abstract artifacts that have longer lifecycles, compared to code and test assets that are highly volatile. And depending on the nature of the business, requirements may be set in code or continually ephemeral. No wonder that the software delivery lifecycle has often resembled a game of telephone tag.

A decade ago, Rational pioneered the vision that tools covering different stages of software development belonged together. But it took a decade for the market that Rational created to actually get named – Application Lifecycle Management (ALM). And it took even longer for vendors that play in this space to figure out how the tooling should fit together.

What's interesting is that, unlike other more thoroughly productized market segments, there has been a wide diversity among ALM providers on where the logical touch points are for weaving what should be an integrated process.

IBM/Rational has focused on links between change management, defect management, and project portfolio management
Borland’s initial thrust has been establishing bi-directional flows from requirements to change management and testing, respectively
Serena and MKS have crafted common repositories grafting source code control and change management with requirements
Compuware attempts to federate all lifecycle activities as functions of requirements, from project management and source code changes to test and debugging
But what about going upstream, where you define enterprise architecture and apply it to specific systems? That's where Telelogic has placed its emphasis, initially tying requirements as inputs to enterprise architecture or vice versa.

It has now extended that capability to its UML modeler and Java code generation tool through integration with the same repository. What would be interesting would be generation of BPMN, the modeling notation for business process modeling, that several years ago joined UML in the OMG modeling language family. For now, Telelogic's Tau can generate UML from BPMN notation, but nothing more direct than that.

In looking at the different approaches by which vendors integrate their various ALM tooling, it's not just a matter of connecting the dots. The dots that are connected represent different visions of where the most logical intersections in the software delivery lifecycle occur. Should the lifecycle be driven by enterprise architecture, or should we drive it as a function of requirements or testing? Or should we skip the developer stuff altogether and just generate byte code from a BPMN or UML model?

It's an issue where the opportunity to play God might be all too tempting. The reality is, just as there is no such thing as a single grand unified software development process methodology, there is no single silver bullet when it comes to integrating the tools that are used for automating portions of the application lifecycle.

This article originally appeared in onStrategies.

Copyright © 2007, onStrategies.com

Tony Baer is the principal with analyst onStrategies. With 15 years in enterprise systems and manufacturing, Tony specialises in application development, data warehousing and business applications, and is the author of several books on Java and .NET.

Mozilla hits back at Firefox 3 quality slur

Mozilla has hit back at claims that multiple bugs in its forthcoming Firefox 3 browser will be ignored in order to meet release schedules.

IDG analyst Gregg Keizer reported that fixes for as many as eight in ten identified bugs would be ready in time for the scheduled release of Firefox 3 (Gran Paradiso) next year. Keizer cited notes from a Mozilla development meeting to support his claims, carried in a story in the New York Times last Thursday.

"We have 700 bugs currently marked as blockers," the notes said. "That's too many. We're asking component owners to set priorities on blockers, as a first pass of what bugs should be Beta 2 blockers. You want it to be about 10 per cent of blockers, or what you can get done in four weeks."

Mozilla defies a "blocker" as a flaw serious enough to justify delaying a release, or at least closer examination.

"We'll be doing pretty much the same thing for Beta 3, which means that something like 80 per cent of the [approximately] 700 bugs currently marked as blockers will not be fixed for Firefox 3. The hope is that by 'fixing the most important blockers' several times, we'll get to a point where we can cut the rest without feeling bad about the quality of the release. And if we do feel bad, we can add an extra beta or two," the notes added.

Mozilla developer Asa Dotzler hit back at the suggestion that Mozilla was compromising on quality in developing the next version of the open source browser.

"That claim is simply horseshit," he wrote in a blog post. "We've already fixed over 11,000 bugs and features in Firefox 3 and now we're discussing how to handle the remaining 700 issues we wanted to get fixed for Firefox 3."

Mozilla developer Mike Shaver said Keizer cited genuine notes but misunderstood the meaning of the term 'blocker'. "It appears to be an honest mistake, since a set of meeting notes did include that prediction, along with other elements that mention other approaches to the Firefox end-game," he writes. "But it’s not our intent to cut Firefox blockers from the fix list against a hard numerical target or fixed deadline."

"The 'blocking' flag doesn’t always strictly mean 'we would not ship Firefox 3 if this specific bug isn’t fixed'. It can also mean 'we should look at this in more detail before we ship'", he added.

Security bugs within the 700 flaw total will be treated as "important". Bugs that impair surfers' browsing ability will also be prioritised. Up to three beta releases of Firefox 3 are planned. The last release was an eight alpha version of the code, released in September. The final version of Firefox 3 is due out some time next year.

The superstructure of Firefox 3 alpha is built around Gecko 1.9, an updated layout engine. The browser is also due to include features held back from Firefox 2, such as the revamped Places system for storing bookmarks and a rejigged history function. ®

Microsoft lets slip Visual Studio 2008

Microsoft on Monday gave developers early access to the next version of its Windows development tools and framework, for the first time tying in both Windows Vista and the upcoming Windows Sever 2008 and SQL Server 2008.

The company has released to manufacturing code for Visual Studio 2008 and the .NET Framework 3.5, while also making code available to subscribers on its Developer Network (MSDN).

Visual Studio 2008 and the .NET Framework 3.5 are the first versions of Microsoft's tools and framework to wrap up a string of infrastructural elements that have been percolating through Redmond and slowly rolling out separately through a series of updates and new products.

The changes simplify data access and programming across Windows Vista and Windows Server 2008 and SQL Sever 2008 - the latter two due next February. Visual Studio 2008 and the .NET Framework 3.5 are also timed for widespread availability next February.

Arguably the biggest change is the addition of Microsoft's Language Integrated Query (LINQ) architecture, created by Microsoft's Erik Meiker, that simplifies programming with SQL object, XML and relational data models in C# and Visual Basic.NET. Syntax added to C# and Visual Basic translate and compile queries to a set of 25 standard query operators, reducing the need to pick a single data model or know all three.

ASP.NET AJAX will come as standard for web development while server-side developers get Windows Workflow Foundation (WF) and Windows Communication Foundation (WCF) templates, with WCF supporting HTTP programming without need for SOAP but adding support for JSON. Expanded web services support in WCF now includes WS-AutomaticTransaction 1.1 and WS-Reliable Messaging 1.1 with other protocols.

For the Windows Vista and Office 2007 family, there are updates to the base class library, Windows Presentation Foundation and Windows Cardspace.

Previously separate tools for building Office applications that take advantage of the Office 2007 interface - regions, ribbon and panes - and extraction of data from Windows Servers are expected to feature in the professional edition of Visual Studio 2008.

Also, Visual Studio 2008 developers can build applications for versions 2.0, 3.0 and 3.5 of the .NET Framework.®

2 comments posted Post a new comment