The New Windows 8 Bootloader

Literally the very first thing you notice when installing and testing Windows 8 Developer Preview is the new boot screen. I personally find it to be very cluttered and unorganized, and generally aesthetically unappealing. However, compared to the decades of text-based boot selection menus that people are accustomed to, reviewers are seeing this as a dramatic improvement.

Some of the interesting things we’ve noticed about the new Windows 8 bootloader:

1. The new look
2. Completely new process of loading an OS
3. Integration of repair tools into the bootloader itself

New Look

Hopefully the designers at Microsoft can get their hands on the new boot screen before Windows 8 goes RTM. That said, Microsoft has explained the rationale behind the redesign as being purely function over form – the new boot screen is part of the general overhaul of Windows to use the touch-friendly Metro UI. Microsoft is claiming that the new boot menu is intended to be a more touch-friendly interface – and you can’t really disagree with that, as the appallingly-oversized buttons and labels make it very hard to miss-click “miss-press” with one’s fingers. However, just how many people will be dual-booting on a PC with a touch-only input device, ummm, remains to be seen.

That out of the way, the new boot menu is more than just a visual overhaul, it’s virtually a complete rewrite of the BOOTMGR/BCD bootloader that was first revealed during the Windows Longhorn/Vista beta program. Some of the changes are a little subtle, but the Windows 8 DP builds have introduced some very tricky concepts, evolving the bootloader from just a selection menu to something rather more complicated.

New Boot Sequence

For reasons we have not yet been able to discover, though we are confident they will be revealed in time, Microsoft has chosen to completely change the manner in which operating systems are loaded once selected from the boot menu. The usual boot process that just about any bootloader goes through is something like this:

With Windows 8, this boot process has been changed completely, and now something more along these lines takes place:

It’s a subtle change as the boot menu is not shown the second time around, but the PC actually reboots after making the selection. We’re not clear on why Microsoft is doing this, but if I’d had to hazard a really wild guess, I’d say it’s to clean up the environment that’s been altered/modified/corrupted by the new boot menu. Basically, it seems that the new boot menu interface has become it’s own mini-OS, and is possibly running in protected mode (vs the traditional real-mode bootloader), and as such, needs to reboot to bring the system back into a real-mode that the Windows 8 kernel can initialize from and bring the system from real to protected mode itself. In short: the new boot menu is more of an OS and less of a boot menu than ever before.

Repair Tools Integration

If you were still in any doubt as to just how different the new bootloader was, and how much more of a complete and standalone OS it has turned into, you need look no further than the new repair tools integration. Windows Vista introduced the recovery center, a WinPE-based environment with some tools for both automated and manual repair, on the setup DVD, Windows 7 (in some SKUs and as of certain revisions) copied the recovery center image to the local hard drive and could, if the bootloader itself has not been wiped out, automatically provide an option of loading the recovery center from disk. With Windows 8, Microsoft has gone a step further, integrating the repair tools into the Windows 8 boot menu itself.

As you can see from the screenshots above, the bootloader no longer contains an entry/link to the repair center WIM image, the bootloader and the repair center have instead been merged together. Still, the same problem with the Windows 7 implementation remains: if the bootloader itself has been wiped (which is the most common problem), the recovery center cannot be accessed. In fact, we see no benefits for this implementation over the Windows 7 method, and can only see the drawbacks of enormously-increased complexity and longer load times.

EasyBCD and Windows 8 Developer Preview

Now for what you’ve all been asking: how does EasyBCD play with the Windows 8 Developer Preview? The short answer is, EasyBCD supports Windows 8 DP. The long answer is rather more complicated.

It seems that out-of-the-box, the Windows 8 “pretty” (and we use that word very loosely) boot screen supports only Vista and up. It will not show Windows XP, Linux, or other OS entries (in preparation for Secure Boot? Perhaps). EasyBCD can be installed on Windows 8 (though it will attempt to download the .NET Framework if you haven’t manually activated it, because Microsoft has done a 180 and biting the hand of the developers that powers it, has disabled .NET 1.0 – 3.5 compatibility out-of-the-box in Windows 8), and can be used to add, remove, rename, configure, and generally do whatever you want with entries and the Windows 8 boot menu.

The catch is that as soon as you use EasyBCD, Windows 8 will detect something along the lines of unsupported changes to the bootloader and cease to use the new boot menu / boot screen. It will revert to the Windows 7-style boot menu, including the text-based interface and the original boot sequence. Now, while we personally prefer the older menu, we realize that the majority of Windows 8 DP users are rather anxious to both keep the Windows 8 touch-enabled boot screen and retain usage of EasyBCD’s wonderful goodiness, having apparently never heard of the saying “you can’t have your cake and eat it too.” ;-)

Now we’re not sure how/why the Windows 8 boot menu reverts to the Windows 7 menu, because in fact, it is possible to force the Windows 8 boot menu to add unsupported entries if you use only very particular bcdedit commands; but as even something as minor and unimportant as changing the order of boot menu entries in EasyBCD causes Windows to use the old boot menu, it seems we’re missing something. Rest assured, this will all be taken care of in due time, though at the moment it seems rather folly to waste time trying to implement support for what is clearly the alpha stages of the new boot menu and the appropriate course of action would appear to be waiting for the next Windows 8 beta build (rumored to be sometime around February 2012).

EasyBCD, when used from within Windows XP – 7, can be used to add a Windows 8 entry to the bootloader in the same manner that you would a Windows Vista/7 entry. So, in short, EasyBCD supports Windows 8, but Windows 8 does not support EasyBCD :)

All joking aside, synchronization objects such as mutexes and semaphores are essential to safe multithreading and are found on just about any platform under the sun. Mutexes and semaphores alike have one purpose: to keep different threads from messing around with bits and bytes at the same time another thread is, keeping your code free of segfaults and memory access violations alike. But that’s about where the similarities between the synchronization primitives on different platforms end.

POSIX-compliant operating systems with pthreads offer additional really neat synchronization primitives not found on Windows, such as condition variables and read-write locks (the latter is now available on Windows Vista+). And Windows programmers have at their disposal automatic and manual reset events, which make designing certain types of multithreaded software incredibly easy, abstracting away much of the hard-core synchronization logic that lies beneath the hood.

We’ve decided to open source two libraries we’ve found useful in transitioning from Windows development to Linux and from Linux development to Windows. The first (and most important) is an implementation of WIN32 manual/auto-reset events for Linux. While there’s nothing WIN32 events can do that POSIX condition variables can’t, the differences between the syntax and usage semantics of both has resulted in entirely different programming paradigms on the different platforms, making it hard for some developers to port code from one platform to the other or even write code from scratch on the platform they’re unfamiliar with.

Enter pevents. pevents is a C++ library (easily portable to C) for *nix platforms that provides an implementation of WIN32 events on Linux, giving developers access to the CreateEvent, SetEvent, ResetEvent, and WaitForSingleObject functions that make them feel warm and fuzzy inside. More importantly and unlike all the other efforts at porting WIN32 events to *nix in the past, pevents also has support for the all-important WaitForMultipleObjects. WFMO is an important concept in multithreaded programming on Windows, and allows a developer to wait in the kernel until one or more events has fired (or, alternatively, until they have all fired) with a single line of code, resulting in high-performance synchronization waits.

While *nix zealots have long maintained that WaitForMultipleObjects encourages bad programming practices, the fact remains that it can be a powerful tool in the arsenal of a good developer… and any claims that WaitForMultipleObjects is inherently flawed as it leads to the loss of events are outright incorrect statements that only those unfamiliar with correct multithreaded programming on Windows would say. With pevents, Windows developers can feel right at home on Linux/*nix with access to WIN32 events in both manual and auto-reset flavors (MSDN explanation for the uninitiated) with both WaitForSingleObject and WaitForMultipleObjects functions.

On the other hand, *nix developers have long had at their fingertips powerful and lightweight locks adapted for the readers-writers problem (Wikipedia overview). ReadWrite locks (pthread_rwlock_t) are powerful objects that can drastically improve multithreaded performance by allowing unlimited simultaneous read-only access to shared variables while only limiting access to one thread at a time for writing purposes. Microsoft has realized the importance of this over time, and with Windows Vista now has support for read-write locks in the kernel (SRW Locks).

However, as very few developers today are free to target only Vista and above, we have written RWLocks for Windows, a library which provides access to three different flavors of read-write locks, with advanced features not found in either pthread_rwlock_t on POSIX and SRW Locks on Vista such as support for cross-process synchronization, reentrance support, and writer -> reader declination.

Both these libraries are released under the terms of the MIT license and hosted on github. These libraries were developed from the ground-up to be as minimalistic, lightweight and fast as possible (though WFMO requires a bit more overhead and can be disabled at compile-time for better performance). Fork, use, and contribute your changes back. Enjoy!

pevents Linux/OS X/*nix

RWLock for Windows

Please note that these are on-going projects still undergoing development and maintenance. WFMO support in particular is in BETA and can be #define’d out if it’s not required.

Find anything wrong? Drop us an email at neosmart@neosmart.net, comment below, or fork the code at github.

As someone that’s been using both C/C++ and .NET extensively over the past years, I found there was one very important point that Sutter glanced on, danced around, and did everything short of actually naming in his talk and it’s that if you’re doing anything remotely intricate or complicated, leaky abstractions in managed languages will bite you in the ass, and end up lowering your productivity, some times (and if what you’re working on is truly complicated, often times) to a point where you’d have been more productive using C or C++ in the first place.

The concept of leaky abstractions isn’t anything new and I’m hardly the first to point out how it can turn a knight in shining armor into a harbinger of doom and destruction. It’s the number one problem fundamentally present in almost any framework, but even more so in managed languages where the framework is all you have, and you’re not allowed to side-step it and build your own foundations to work with (p/invoke and interop aside). But lately it’s becoming more and more of a problem as the “push” for innovation that Sutter speaks of has become a fundamental requirement in just about all corners of the industry.

5 years ago, few in the .NET community could tell you what p/invoke was and how you’d use it. Now, it’s considered fairly basic knowledge and a working familiarity with the underlying C WIN32 API is a must for any desktop software developer looking to make a memorable, high-performing product in the world of .NET. Pretty much each and every WIN32 API documentation page on MSDN has a comment from someone on how to import that particular function into .NET with an interop definition, and people are running into the limitations of the managed framework far more often than they used to.

I’m still a big fan of .NET in general as it makes it really easy to quickly jump from idea to prototype to finished product for quick, one-off applications or basic tools and utilities. Even for huge products and projects, .NET’s GUI tools far exceed any C++ offerings in terms of ease-of-use and even functionality when it comes to having standard controls and features a single click away. But the leaky abstractions in .NET are proving to be a real pain in the back, and sometimes even interop just doesn’t cut it for the most trivial and basic of things.

Recently, I hit a wall with .NET in attempting to make EasyBCD support the parsing of internationalized BCD output in the form of bcdedit’s stdout. bcdedit, like many other Microsoft tools and utilities, is only UTF8 or Unicode aware if the console codepage is explicitly set in the application’s console window. I spent an entire day trying to shoehorn this functionality via various .NET hacks and failed interop attempts, a week off-and-on testing and debugging my various incomplete solutions, and in the end my solution was to create a C++ “proxy” application that would create a console, set the codepage to UTF8, run the command line utility, and pipe the output back to the host (source code here, MIT license: UtfRedirect). It was a guaranteed fix, took all of half an hour including testing it on 4 different platforms and experimenting with various internationalized stdout texts.

I personally believe the “best” compromise for medium-sized projects is to use C#/.NET to create the GUI and small helper scripts/utilities, but to build the core in C++ with an exposed C API or commandline interface. The short and long of the matter is, making a language truly and properly productive is a lot more work than just providing a sandbox with prettied-up API calls. You have to organically add improved productivity into a language from the bottom-up, making sure that the productivity goes hand-in-hand with power and flexibility, that it always second fiddle to being able to get the job done; after all, what use is a one-liner code if no mater how you twist and turn you can’t get it to do what you want?

What’s different with Windows 8 and Internet Explorer 10 is that Microsoft chooses to deflect the blame from itself, and pin it instead on the unfortunate owners of the website that the user had last visited. In the following screenshot, Internet Explorer 10 has crashed after being unable to handle some of the web scripting on our website. Except instead of the error dialog you would expect (something to the effect of “Internet Explorer has stopped responding,” an error caption which we are all familiar with and of which the web is full), the error dialog instead reads:

neosmart.net is not responding.

Come again? Really? To us, this seems like a rather mean and dishonest approach to dealing with the shortcomings of the web browser. While it is true that at this moment in time, Internet Explorer is more prone to crashes and hangs than it would be in the final (RTM) release of Windows 8, this does not in any way give Microsoft the right to assume that in the future any hangs or crashes in IE are to be blamed on the designers/coders of the website being visited.

In fact, no code even if purposely meant to crash the browser should succeed in doing so, and certainly should not be held to blame. If Microsoft cannot make a crash-proof browser, that does not mean that they should dishonestly deflect and redirect the blame towards and innocent third party.

Almost four years ago, NeoSmart Technologies published a Windows Vista repair and recovery CD that could be used to treat common boot issues and recover from catastrophic system failure in case you didn’t have a Windows setup CD handy.

Over the years, we’ve added more recovery CDs to the collection, ending up with a complete portfolio of repair CDs for Windows Vista and Windows 7 in both 32- and 64-bit flavors. We’ve had the good fortune of being able to host these CDs on our site in one form or the other for free download to millions of users around the globe.

Three months ago, we were contacted by the legal department at Microsoft Corporation asking us to discontinue hosting these files. Until this point, we were not aware that Microsoft was displeased with our hosting of the CDs and in fact enjoyed a rather healthy relationship with the Microsoft support forums where victims of PC crashes would be directed to our site to download a copy of the repair CDs.

Since then, we have been in talks and negotiations with the Microsoft legal and licensing divisions, trying to work out a method whereby we could provide our users and visitors with access to these CDs once again. Today we’re excited to announce that these CDs are once more available for download!

There’s some mixed good and bad news, however. The good news is that all downloads will be direct HTTP links, so no need to learn how to use a torrent client or fiddle around with open network ports: just plain, standard, direct, and very fast HTTP downloads for all our ISO images. We’ve also taken this opportunity to update the repair CDs and add a virus scanner (powered by ClamWin) as well as some command-line tools and utilities that were missing from some of the images.

The bad news is that there will now be a nine dollar download charge for each CD. There was no way for us to avoid charging this, as we now pay a licensing fee to Microsoft in exchange for making these CDs available, and also pay per download to outsource the high-speed web hosting for these large CD images (150 to 200 MiB, each) to an outside service.

We look forward to updating these repair and recovery CDs over time and adding more tools and utilities to help you save your PCs from the brink of death. We thank you for your patience with us over the past few months during which the downloads were suspended, and truly appreciate your understanding of the situation.

Download Links

Windows Vista Recovery Discs (x86/x64)

Windows 7 Recovery Discs (x86/x64)

There were issues pertaining to dual-displays in a mixed VGA/DVI environment (one display DVI, the second VGA) which were never resolved by ATi (to the best of my knowledge) and were worked around by initially downgrading the DVI to a VGA connection and later replacing the older monitor with a new DVI-based display. There were issues related to the refresh rate. There were issues related to the resolution. There were issues related to the games. There were issues related to the OS. Basically, wherever you look, there were problems caused by poor development practices and crappy QA all around.

There were issues that Microsoft/Windows’ new WDM model caught, triggering a restart of the graphics subsystem without causing a BSoD. And there were (unfortunately the majority) of the ATi display driver crashes that caused BSoDs left, right, and center.

Today, my Windows 7 PC surprised me with an interesting question:

ATI display driver has stopped working properly. [big surprise!]

At this time we don’t know the exact cause of this problem [again, no surprises here!]. Hewlett-Packard [!!!] is interested in gathering additional feedback. Please visit the Hewlett-Packard website to provide information about the problem.

Pin-drop silence. Remember, this PC is homebuilt. HP has nothing to do with it. If it said “report to ATi,” or “report to AMD,” or “report to Microsoft,” I’d understand. But HP?

And it wasn’t a mistake. Following the link presented takes me to this HP page. And the first question? Did this crash occur on an HP PC?

So it’s not a mistake on Microsoft’s behalf — HP is making a real effort to track down a bug caused by ATi drivers running ATi hardware. Wow.

All I know are two things:

1. Having been in contact with ATi over these problems for several years, I’m glad someone else is looking into it now.
2. I’m not getting another ATi/AMD graphics card ever again!

However, there are a few things that have forever irked us about the ln lookalike called mklink.exe:

• It’s called mklink and not ln. (I mean, you just get can’t get around that fact)
• The arguments are switched around. `mklink something_doesnt_exist actual_file` is just…….. wrong!
• By default, mklink will create softlinks and not hardlinks. ln requires the /h flag to create a hardlink.
• mklink isn’t smart enough to distinguish between files and folders. You need explicitly tell it via the commandline.
• Even then, mklink has two different switches depending on the type of directory link you want. /D for softlink’d directories, and /J for hardlink’d directories.
• mklink can’t be used outside of cmd.exe (such as in PowerShell). (Hat tip: Jason)
• And, of course,  mklink isn’t open source.

So we made our own.

ln-win is available in 32-bit and 64-bit, uses the same syntax, names, switches, and argument ordering as ln on *nix. And pursuant to our promise to contribute more to open source, ln-win and our Junction Point library is 100% open source, freely licensed under the generous terms of the MIT license.

Perhaps more importantly, ln-win contains an open source Junction Point library. While Win32 APIs exist to create symlinks (on Vista and up), there is no API for creating hardlinks of any kind (for files or folders). The NeoSmart JunctionPoint library provides the following APIs:

	bool CreateJunctionPoint(LPCTSTR origin, LPCTSTR junction);
	bool IsDirectoryJunction(LPCTSTR path);
	bool GetJunctionDestination(LPCTSTR path, OUT LPTSTR destination);
	bool DeleteJunctionPoint(LPCTSTR path);

As you can see, the NeoSmart JunctionPoint library provides all the APIs you need both to create and interact with hardlinks. IsDirectoryJunction can be used for both symlinks and hardlinks, making it a snap to make your code symlink/hardlink aware, letting you intelligently avoid cyclic paths and data duplication for any filesystem traversal needs.

Binaries for Windows x86 and x64

ln-win Source Code on GitHub

We’d previously finished the tutorials for dual-booting Windows 7 with Windows XP and with Ubuntu 10 (complete with picture-by-picture steps!), but now we’re getting started on the real meat: the technical details of just what exactly is going on behind the scenes. The normal OS boot process is one of the most complicated parts of an operating system with just one OS in the mix – with multiple operating systems, each that works in its own way, things get that much more complicated, and it’s always good to have a nice, illustrated guide to refer to.

The first part of the updated documentation is now available in the form of an explanation of how a dual-boot between Windows Vista/7 and Windows XP normally works, plus a NeoSmart exclusive: EasyLDR and its support for multiple top-level legacy NT entries without needing to go through a second NTLDR boot menu, a long-awaited feature that was finally added with EasyBCD 2.0.

EasyBCD Documentation: The Windows XP Boot Process

EasyBCD 2.0 features a number of exciting new features, but some of the most technically interesting details lie in the improved Windows XP support. Besides the linked article which explains the mysteries of EasyLDR and how EasyBCD accomplishes top-level Windows XP support, a large amount of research went into the creation of BootGrabber.exe, a command-line application that we wrote to automatically configure everything needed to set up easyldr or ntldr, complete with automated detection of all Windows NT – 2003 installations on mounted disks, the generation of a corresponding boot.ini/ebcd.00x file, the copying of ntdetect.com and boot.ini to unmounted and unlettered partitions, and changing the active partition.

Hopefully another article will be added really soon to the EasyBCD documentation explaining the powerful command-line features available to users in BootGrabber.exe, which may be accessed via the “Power Console” under “Useful Utilities” in EasyBCD 2.0 and up.

The changes in EasyBCD 2.0.2 correlate to 21 issues in the bug tracker, and can be seen after the jump.

Download EasyBCD 2.0.2 (1343 KiB)

Alas, with this update the EasyBCD setup is no longer 1337 KiB. But that’s OK, we figured you guys would rather the new features and bug fixes over a really, really cool download size. And here are the changes. (Important changes or high-frequency bugs are in red, and the complete changelog to date can be viewed here)

• [EBCD-385] – EasyBCD will crash if an operation that it expected to take a long time finishes instantly
• [EBCD-386] – EasyBCD crashes if you attempt to save entry settings with no entry selected
• [EBCD-387] – Cannot add multiple WIM entries
• [EBCD-388] – EasyBCD crashes if Segoe UI is corrupt
• [EBCD-389] – EasyBCD options dialog crash on international computers that don’t support mm\dd\yyyy format
• [EBCD-390] – EasyBCD crashes if an invalid path to a WinPE image is specified
• [EBCD-391] – EasyBCD doesn’t show actual DEP, SafeMode, Pae in Advanced Settings
• [EBCD-392] – EasyBCD crashes if an invalid path to a VHD image is specified
• [EBCD-393] – Improved boot.ini partition detection
• [EBCD-394] – EasyBCD crashes if corrupt WinPE entry with no ramdisk is present
• [EBCD-395] – Use BootGrabber to set active partition instead of relying on diskpart
• [EBCD-396] – BootGrabber /setactive switch
• [EBCD-397] – Try to get partition information from bootsector, not just partition ID
• [EBCD-398] – EasyBCD crashes if adding a WinPE or VHD entry fails in the AddWindowsVista stage
• [EBCD-399] – Crash removing NeoGrub immediately after adding it
• [EBCD-400] – Select correct disk for boot-partition related operations when editing external BCD
• [EBCD-401] – Update GRLDR to 0.4.5b-2010-07-25
• [EBCD-402] – Crash if bcdedit output contains null character
• [EBCD-403] – Handle IO Exceptions in creation of NTLDR/EasyLDR
• [EBCD-404] – EasyBCD crashes if bcd timeout is greater than 10000
• [EBCD-405] – Ignore exceptions when deleting stub loaders

Special thanks goes to joevt for his help and patience in fixing EBCD-393.

Hello and welcome to EasyBCD 2.0!!

It’s rather hard to believe, but EasyBCD 1.7.2 has been out for over 2 years now, and we’ve been working on Version 2.0 ever since. In that time, a lot has happened. Windows 7 has shipped, ext4fs is the new cool kid on the Linux block, GRUB2 is finally seeing some adoption, VHDs are the new wow, and everyone and their grandmother want a dual-boot between Windows 7 and Windows XP.

Worry not, we haven’t been sitting on our (not-so-proverbial) behinds this whole time. In fact, the entire NeoSmart team – developers, supporters, testers, and all – have been working around the clock to make EasyBCD 2.0 the biggest, coolest, greatest, and awesomest thing ever since the invention of the MBR. And now, over a 150 beta builds later and 2 years in the making, we’re super-pleased to introduce you to EasyBCD 2.0. It’s so incredibly overhauled and improved, so stuffed-to-the-brim with features, so much of a true one-click dual-boot experince, so customizable, so powerful, and so EASY that it took a lot of self-restraint to keep from calling it EasyBCD 10.0!

What’s new, you ask? We’ll get to it. But let’s just first give you the download link, because we know you just can’t wait to get your grubby, geeky paws on it ASAP: