The Basement Coders Developer Podcast

On my Windows 7 box, I added some hard drives. The formatting dialog box offered the option of formatting it as NTFS or exFAT (aka FAT64). I chose NTFS but was curious as to what exFAT. One wikipedia article later, I was enlightened: it's basically and propriety and patented update to FAT useful for large disks and large files. As portable disks/flash cards are getting bigger (to support video cameras, etc), a suitable format had to replace the venerable, but limited, FAT.

Microsoft, and its defendants, have been claiming how reformed the software giant is, and is more open (with respect to interoperability and compatibility with competing systems and companies) and plays well with others now. Samba and its office formats are getting more open. The world is generally a better place, supposedly.

Except it seems the right hand of Microsoft doesn't seem to know what the left hand is doing.

The most basic, fundamental way to interoperate with an OS is via shared disks. Floppies in the past, hard disks and flash drives now. If you want to interoperate with a PC desktop, that means interoperating with Windows, which means you can only choose formats it supports out of the box.

Microsoft, by virtue of this monopoly, gets to decide the successor to FAT. Making a lower-overhead, but scalable file system suitable for removable disks isn't hard. There are many free and open solutions out there, and coming up with one from scratch isnt innovative or challenging, especially for Microsoft. So they did just that and called it exFAT.

The problem is, the new Microsoft has decided that it wants to cash in more on its technology licensing and patenting (see the story on the Register), and has decided to heavily license (charge a pretty penny for) exFAT users. These users include flash disk makers, camera/device makers and of course, competing operating systems. Linux in particular is going to be in a grey area, as its hard to extract per-user fees out of something that's often freely downloaded.

exFAT isn't impressive technology. It's not innovative or game changing, and not worth licensing for the technology itself. If anyone else came up with it, it's be ignored and forgotten. But since it's Microsoft, people will have to use (and thus pay for it) to be able to talk to Windows. To use an analogy, the electric company is charging vacuum makers to make cords that can plug into their sockets.

Microsoft, yet again, is leveraging their desktop operating system monopoly into other areas and profit centers, stifling competition. The leopard can't change its stripes, despite its PR. Is anyone actually surprised?

Slashdot recently ran a story on yet another "Why Linux isn't ready for the desktop."

This one was rather nice. It contained a nice technical analysis with only a few errors. It's not this one that I'll be complaining about today, it's the millions of others of these types of articles that seem to be written by long-time Windows users that just happened to have tried Linux and need something to blog about.

Most of the time these users list install issues or driver support as their biggest issue, then conclude that because Photoshop or some other specific applications aren't available (and assume never will be) that Linux is doomed and its users should just give up.

Unfortunately, these arguments are flawed on several fronts. First, Linux can't do anything about the lack of availability of certain commercial applications. This is just a fact of smaller market share. As market share increases, these applications will come on board. Linux can be the perfect operating system, but if it only has 1% of the market share, your specialized accounting application that you bought five years ago still isn't going to run (ignoring wine and virtualization solutions here). Linux is sometimes able to offer you alternatives (for example, the GIMP), but if you really need Photoshop then yeah, Linux isn't an option to you. Only Adobe can rectify this.

Second, no one ever has to install Windows - it comes with their computers. The hardware maker has already made sure everything (hardware-wise) works out of the box. Testing and judging Linux's install process but not Windows' seems unfair.

As a full-time Linux user and programmer, I can't ignore Windows and still must boot it up occasionally to compile and test my software. Every time I go back to Windows to do this work, I'm constantly reminded of the flaws Windows has, flaws that are probably ignored and invisible to the veteran Windows-only user. In fact, the flaws are so glaring, that I sometimes wonder how this crappy operating system became (and stays) the leader (answer: momentum: PC builder support and 3rd party software support). After I'm done a typical session, I snidely conclude that Windows (XP in this case, lets not get started on Vista) isn't yet ready for the desktop, and here's why:

Poor installer - not usable by average users (luckily they don't have to). Lacks drivers and poor auto-detection of hardware, often leaves the computer installed without network, sound or 3D video card support. Had to go to another computer with network access to download network card drivers for the first machine. Have to search the backwater download areas of various sites, running random (and dangerous) executable installers hoping that I have the right one for my particular hardware adapter. Lots of reboots between driver installed wasting boatloads of time. Older hardware often completely unsupported in new versions of operating systems. No live CD. Unfriendly to non-Windows partitions.

Auto upgrades seems slow. Asks a lot of questions, and is very intrusive. Tries to sneak in and install additional software (WGA, IE, etc). Sometimes forgets previous refusals and nags some more. Can scare new users easily, especially when asking about installing new software or displaying EULAs. No nice (central, Internet) package management system. Add/remove programs panel in Control panel sometimes doesn't work and doesn't include all software. Updater sometimes reboots the computer automatically, abruptly terminating any running programs that the user may have delivery left running.

No central upgrading system for 3rd party software. Why doesn't the add/remove program window in control do this? 3rd party (or even 1st party Microsoft like Media player) seems to do their own updating, constantly nagging the user with update and restart requests. New users can be surprised by these queries. Highly intrusive. Upgrades can easily break stuff (no proper dependency tracking) as upgrades may not just include version patches, but completely new versions of software. Not user friendly at all.

Requires virus scanner. Base operating system demands it, yet doesn't provide one. Can be disabled though. Virus scanner makes computer much slower. Without such a scanner though, machine is highly vulnerable, especially since many required utilisation and software packages come from the Internet via the web browser as executables - not the safest delivery vector. Some kind of chmod +x system should be required - software should come in packages and/or package repositories.

No nice repository for additional software. Must find them developer's web sites. This however can be dangerous as if you aren't familiar with the developer, there is a risk of getting malware. There is some kind of signing system for executables, but the user can (too) easily dismiss the queries/warnings, rendering the system ineffective.

The default web browser, IE, is still quite insecure, has (had) poor web standards support and still supports ActiveX, an insane idea (from a security standpoint).

Lacks lots of basic software. PDF viewer, ssh clients, GUI ftp client, torrent client, decent zip-client, decent photo editing, CD/DVD burning software, CD-to-MP3 ripping, etc. Some stuff can be downloaded from Microsoft's site, but this requires knowing about it, finding it and other hassles. Sometimes add-on software can wreck havoc, like after-market firewall or wifi network management software. New users would need the advice of experienced users to avoid installing malware, who have set up realistic-looking mock sites to trap and ensnare users.

Lacking lots of codecs out of the box (like DivX) with no obvious method of getting them. Media player won't rip to MP3 by default, insisting on WMA, even though MP3 is the de-facto standard (to be fair, iTunes is guilty of this too)

The basic software that is there is not polished. The picture viewer doesn't do EXIF rotation. Notepad doesn't support UNIX or mac line endings. WebDAV client support is just broken.

Very unfriendly environment to developers or system admins (and other such power users). Poor command line (shell), no ssh system, non standard build tools (like make), no scripting languages out of the box, lacks many basic command line utilities. This would seem easy to fix, but Microsoft seems to like to make Windows drastically different (i.e. dislikes anything command line related or anything it didn't invent) in this respect. This results in Windows being a difficult platform to use when working with many open source projects or languages. Developers pretty much have to be using full IDEs and packages, but even in those cases, assembling libraries and build systems is still a pain with lots of trial and error. Cygwin is a half-solution at best. A Microsoft sanctioned shell, ssh client/server and collection of utilities would be best. Apple seemed to have done it, why can't Microsoft?

Not very customizable or tunable by hardware makers, making it an ill fated choice for netbooks. Depending on how the netbook market goes, this may or may not resolve itself. If netbooks get more powerful for the same price, then they can more easily handle Microsoft's operating systems. However, if they get cheaper by using even lower cost components (ARM CPUs, for example), then Microsoft's operating system will be completely shut out.

Windows, on the surface, seems like it would be a complete failure as a desktop operating system. However, it has the world's momentum behind it. All system builds prep their products for it, all corporate technicians are trained to care and feed Microsoft's offerings, a large software ecosystem is ready to fill its gaps and everyone has just plain gotten used to its quirks. This tolerance for crappy software is often misconstrued as being easy to use, when in fact it's simply the devil-you-know vs the devil-you-don't phenomena. People don't so much think Windows is easy to use as they're just used to its crapulence and don't want to bother with anything else. They might even know it's a crappy system, but at least they're comforted in knowing that their system isn't any crappier than their neighbours - everyone's in the same boat.

Suffice to say, there will be no distinct "year of the Linux desktop". It'll be a gradual process, as Linux slowly grows into markets (netbooks, corporate desktops, servers, devices) and slowly encroaches into general desktops. Linux lacks all kinds of software, but every year, it gets a few more (free and commercial) apps (or the apps move to the web) and a few more converts.

Recently, the Scala language was brought to my attention.

Now, I've been around the language block a few times, having seen my share of languages. I'm aware and suspicious of the siren-like call new languages have for programmers. They tickle our programming curiosities to the core, offering silver-bullet, paradigm-shifting opportunities for increased productivity. Unfortunately, they require lots of commitment, community, support libraries and momentum to become useful, so one has to be pragmatic when adopting new languages. I tend to error more on the conservative side, but Scala beckoned me to have a closer look. These are my cursory observations.

What first drew me to Scala was its offering of type-safe and concise programming models, based on generics and functional programming. Although the key reason I use C++ is performance, it's the type safety and concise power of templates that keeps me there. I find type-safe languages provide for more readable and maintainable code bases, something that is critical for large and aging projects. Scala seems to push for some of these same ideals.

The first benefit of Scala is immediate: it compiles to Java byte code and is able to utilize Java libraries. This means you can integrate with your existing Java code, use the plethora of existing Java libraries and reuse the man-centuries of work that went into Java VMs. This is somewhat analogous to how C++ can use C. However, C code can (mostly) be compiled as C++ code, Scala code is not anywhere near valid Java code. Although this affords the Scala designers more freedom, it creates another learning curve for Java programmers.

I may have overestimated Scala's intended integration with Java though, as they also have a preliminary link up with .NET working. This could be interesting, or it could split its community focus.

The language itself is very functional and recursive. Functions are first class objects, specialization/type searching is handled via a powerful pattern-matching system and generics seem to be on the right track. This kind of stuff lets you build very interesting DSLs (domain specific languages), something that I think will vital for the future. This means you can build stuff like C#'s LINQ SQL query system into a programming language, without having to update the language -- all done as a library. New web presentation mark-up languages, mini-programming/scripting languages, XML query/processing, etc. could all be done as libraries. Neat stuff.

My very cursory inspections with "javap", the Java class disassembler, shows that Scala is not doing the type specific in-lining I thought it would be (for example, instantiating functions for each specific type T that is used). This means I won't get my performance wishes out of Scala, which is too bad, since that would be more interesting that providing yet another deferred-to-runtime type system (even if it has a nice type-safe compile time checker).

It's an interesting project, and I wish them the best of luck. Programming languages require large investments in time, community and libraries, and they're just beginning this long road and still have a ways to go before I'd commit any large projects to them. This segways into my final, not-really-Scala observation:

Why are the Twitter guys all hot for it?

Twitter (via programmer Alex Payne) seem to be claiming that their notorious problems with scalability and reliability can be traced back to Ruby on Rails. RoR haxxor Obie Fernandez attempts to refute this. Whether RoR can handle the load or the Twitter programmers just don't know how to use it, I don't know or care. What I do know is that if you're going to switch to something else, especially for your critical messaging queuing core, don't jump to something even more obscure and untested.

Alex Payne says Scala is fun and gives him the fuzzies, which is all fine and dandy, but for god's sake man, think about the business here and not the Scala book you're penning. You need something scalable and tested, which usually means old and boring. That's just business. Make Twitter work, sell it, and get (even more?) rich.

I recently came across this C++ Soup post, which demonstrates some cute C++ tricks. Now by cute, I mean stupid, similar to Dave Letterman's segment "stupid pet tricks" (you can google that if you've never seen this). It's a useless language exercise with little practical application except to show how obfuscated one can make C++. Are there any benefit to these tricks? Lets explore.

Now all C++ programmers do this (myself included) just to see if they understand template programming, so keep in mind that this is NOT a critique of this particular author, who I'm sure is a C++ Ninja. It's just a recent example that triggered this rant. No, just have a look at half the boost libraries to see this thinking taken to the extreme. Or even the C++ standard library; does anyone really use the cumbersome std::for_each function or any of that bind stuff? They all feel like they're unfinished ideas: they're onto something, but they're not quite there yet.

These (what I'm calling) C++ tricks are often segments of templated C++ code that push generic programming to solve some kind of basic problem, that is typically solved using simpler (but more verbose) code. A trick author is often amazed (and proud) at how small or flexible the code is, but hopefully realizes that the amount of work or expertise required to make the trick outweighs most benefits. The code is then shelved. Lets cut this silliness out.

Furthermore, most application developers will steer clear (or be ignorant) of such tricks, and popular C++ libraries do well to not require such expertise from their users.

Is there any benefit to these tricks? Yes, only to show what could be possible with C++.

C++ is an extremely powerful language, giving you compile time dynamic/generic programming with no loss of runtime performance. No other language does this so fully and completely, and C++ is still the only language option in many demanding areas.

However, the language needs to grow, and it needs to grow fast. We must look at what these tricks are trying to do, and then think how we can change the language to do the same thing in a sane, logical fashion. C++0x is working on this with lambdas, type inference, better error reporting and some other niceties. But we needed these features yesterday, and get people using them. Lets steal more features from other languages. Lets turn up the meta programming capabilities so we can extend the language without meta-compilers and preprocessors, which until now provide the bulk of the useful new language extensions.

There is no reason why C++ can't offer easy to use and powerful compile-time dynamic programming, resulting in code that looks like a dynamic scripting language, but with safer types and kick-ass performance, memory use and code size.

Lets get to work

The Linux Outlaws Podcast recently (in ep. #75) had an interview with Miguel de Icaza on the topic of Mono. Since this is often a heated topic among Linux hackers, I figured I'd chime in with my two cents.

Miguel is well know in Linux land, having started the GNOME desktop and countless other desktop applications for Linux. He's done more for Linux in the past decade than most (including myself) will ever dream of doing, so critiquing such a leader can't be done lightly.

Miguel unfortunately started out as a typical C-only guy who looked on C++ as large and useless. For years with GNOME he and application developers (myself included) slaved away using the C-based GTK+ library to make graphical applications. Man-centuries were wasted dealing with cumbersome C and its unforgiving API style. C++ was ignored, almost laughed at, despite the fact commercial C++ GUI libraries have been the norm. To this day, GTK+ is still an utter failure with respect to commercial application adoption.

Eventually, even Miguel realized that C was not the language for desktop application development as it was hurting GTK+ adoption. Rather than embrace C++, he threw out the baby with the bath water and embraced C# itself for GNOME application development with the Mono project.

The Mono project is an open source implementation of Microsoft's C# language and runtime for Linux and other operating systems. Although one goal of this project is very honourable (allowing custom/vertical/boring applications from Windows-centric shops to be usable under Linux), I think in general Microsoft is making a sucker out of the Mono developers and users, and here's why:

Microsoft was once an innovative, competitive company. That however, was along time ago. They're now a convicted monopoly who still to this day function primary to lock people into their various technologies. I don't want to sound like a typical "Microsoft is evil" parrot but this is what a monopoly does. I can't fault them for it, but let's not be naive. I can spend a blog post alone on Microsoft's various transgressions, but for now I'll simply rat off areas which they really like to maintain their monopoly lock-ins: file servers/smb, office formats, web browsers/ActiveX, web browsers/IE stagnation, media formats/codecs, flash/Silverlight. Heck, I can't even access my company's MS Exchange web interface in Firefox without getting a substandard interface to email. They can't (well, don't want to) even make a website without trying to lock-in you into Internet Explorer on Windows.

Microsoft is not stupid and understands that platforms are really about capturing and holding developer mind share. Java and similar truly multi-platform technologies that they didn't control was viewed as a huge threat to the Windows empire. So Microsoft "embraced and extended" Java-principles, gave it a new name (C#) and injected a boat load of Windows-specific stuff into it. We can debate the finer differences between the languages until the cows come home, but fundamentally, at their core, they're identical languages, except that C# isn't multi-platform and is very Windows-centric. By the way, Adobe's flash/Actionscript is the next threat and target after Java, so look out for a serious assault on that front from Silverlight.

Given this, it's easy to see why Microsoft likes the Mono project. They get to pretend C# is multi-platform ("see, we're not a monopoly!") and friendly towards Linux and open source, but they don't have to devote any of their own resources to it. Furthermore, Mono on Linux will never be a first class citizen (like in Java) and will always be a version or three behind, lacking in proprietary features. To get the "true" C# experience you'll still be sold a Windows system. Microsoft gets to point to Mono and claim that C# is multi-platform, but will then tell you you must run their version on Windows to get the true, modern and full experience. This basically relegates Mono to (using the colloquial term) being Microsoft's bitch.

Although I used to code a lot of Java, I'm not really a Java proponent. But honestly, if the Linux desktop needs a garbage-collected, byte code-interpreted, type safe and object-oriented language, then Java is hard to beat. Decade or two of testing, mature IDEs, plethora of libraries and truly multi-platform (first class citizenship for Linux) and now open source (I hear?) are all features that would be stupid to ignore. But for some reason, Novell decides to chase Microsoft's ever changing specs while having to rebuild everything from the ground up.

Miguel ends off the interview with what I think is a little Stockholm syndrome. He says that most people unfairly demonize Microsoft, ignoring the specific points (like the above) those people make. Heck, he excuses Balmer's public demonization of Linux as the spouting of a sales man, ignoring the fact that this is the CEO of the big ship Microsoft. One simply needs to use non-Microsoft operating systems and applications in a Microsoft-based shop to truly understand the breadth and effect of their monopolistic lock-in.

But alas, I realize that there are other factors here in Novell's decision to fund Mono. Novell is scrambling for money and business opportunities, looking for any edge against Red Hat, even if that means dancing with the monopoly Microsoft. Although you can't fault Novell for doing what they think is in their best business interests, let's not get delusional into thinking Mono is on some level playing field with Microsoft.