This article is only available in Finnish. It relates to my talk at Tech Days Finland 2012.
8.3.2012 pidetyssä Tech Days 2012 Finland tapahtumassa olin puhumassa UI Kompositio aiheesta, miten rakennetaan käyttöliittymää käyttäen Javascriptiä pienistä paloista. Tästä löydät esityksen PowerPoint sekä PDF muodossa, ja demo sovelluksen.
Kun lataat demo sovelluksen, muista lukea asennus.txt tiedoston mikä kertoo miten tietokannat ja ympäristö rakennetaan.
UI Komposition esitys (.pdf)
UI Komposition esitys (.pptx)
UI Komposition demo (.zip)
UI Kompositio
(ASP.NET)
Windows 8 Consumer Preview
(Windows)
The Consumer Preview version of Windows 8 (fancy name for public beta) became available yesterday. I installed it into a virtual machine. This article summarizes my first impressions. I will updating this as I explore more features of Windows 8. My initial very negative opinion on the UI is now not so negative, resulting from the exploration and learning I did. But this also means that there is a learning curve, without it, with just intuition alone, you will not be able to "enjoy" the new UI, in my opinion.
Overall, apart from some usability complaints (which are explored in the rest of this article), I could probably imagine working in Windows 8. The UI still feels strange and a little bit clumsy. Probably this is because I do not think the Metro style UI and applications are in harmony with the desktop paradigm. Anyhow, there are improvements compared to the developer preview, but I still find it is not as smooth of a user experience as Windows 7 is today.
Update: If you are a heavy keyboard user, here is a list of keyboard shortcuts that come in very handy with Windows 8. Very useful! Windows 8 keyboard shortcuts.
If Windows 8 would be released today, I do not think I would want to upgrade. The mix of Metro-style and regular applications is not as smooth as I would want it to be. Frankly, I do not think this new Metro-style thing belongs to the desktop, and I for one do not see as an improvement.
Features of Windows 8
I will not try to introduce the features of Windows 8, because there are tons of articles with pictures. Please refer to Engadget for one such article. And I really like the punch line for that article:
We really liked Windows 7 when it launched. It felt like a big step forward in the short time that had passed since Vista. Now, as we creep closer to a likely release near the end of this year, we can't shake a sense of doubt. Windows 8 still feels like two very different operating systems trying to be one. The potential is hugely alluring -- a single OS to rule both the tablet and the desktop -- and with each subsequent version we keep hoping this will be the one that ties it all together. Sadly, as of the Consumer Preview, we're still seeing a lot of loose threads.
As it stands, Windows 8 is a considerably better tablet operating system than any previous version has managed to be. However, it's still a clumsier desktop OS than Windows 7. That's a problem Microsoft must fix before release. (from Engadget)
The Metro experience
Anyone reading about or trying out Windows 8 will probably know what Metro-style applications are. They can be started from the new Start screen, they run full screen, and cannot be removed from being full screen (other than using snapping). Running applications full screen seems natural for certain solutions. For web browsers, it is a natural way to operate. For some other applications, it just isn't.
Sidenote: Why kill windows in Windows? If everything is full screen, there are no windows anymore. Hmm?
Full screen
For some applications, running them full screen seems outright ridiculous. For example, take the default calendar application on a 24" screen. On a 10" tablet those might look nice, but it just does not work on big screens. Messaging application - I never-ever run chat/messenger in full screen. I like having it floating around above/near other windows. I can do work in my main window and follow the chat at the same time.
I have created some tools and small programs that run as quite small Windows. There would be no sense in making an application full screen, that requires 2 or 3 textboxes for input. I simply cannot imagine how such a tool could be reimagined into being a Metro-style application. Of course if I wanted to create a smartphone or tablet version, sure, no problem. But putting 2 textboxes and a small image on a 24" screen? Or 30" for that matter. I just do not think this is a universally good idea. (I am about to explore Windows 8 application development further, so I might revise this paragraph in the future)
You cannot switch Metro-style apps to run in a Window. Going back to the calendar application: I usually have a webpage, email, chat or something else opened to which the calendar would relate. Having windows side by side makes it much easier than having to alt-tab all the time. I have a high resolution 24" monitor at home, and two large displays side-by-side at work. I have them not because I enjoy looking at full screen things. I have big screen real-estate to actually use it for MULTIPLE WINDOWS at the same time. Forcing new apps (Metro-style) to run full-screen, it is a waste of screen real estate.
Snapping applications partially solves this problem. You can use Windows+. and Windows+Shift+. keys to snap application windows. You can even snap a Metro style app (the calendar) and then the remaining space can be used by the desktop. And mind you, snapping is not something you would figure out for yourself - I certainly didn't
Normally I can remove apps from full screen, and very easily put them side by side. Just press "Windows+left" and "Windows+right" and you can arrange your apps very fast in Windows 7. Or use the mouse and drag the title bar to the left side, top side or right side. Very very easy and very very fast. I was actually very impressed, how much thought went into the usability of Windows 7. With Windows 8, I do not find that Metro-style apps improved this in any way, even considering snapping.
I really do not care how much Microsoft tries to justify this new user interface. When Windows 7 came out, you could feel the improvement. With Windows 8, you can feel the setback in user experience.
When you look at the changes from the tablet perspective, it of course makes all sense. I can understand ALL of these choices and behaviors when using a mobile phone or a tablet. There I would even expect everything to work like Metro does. And I don't doubt that Windows 8 tablets will be very nice to use. BUT the desktop is different, and using the same paradigm for the desktop will hinder performance and efficiency.
Messenger
I seem to be signed in to messenger as soon as I sign in to Windows (because I used a Live account to sign in). Where do I began to list all my problems with this solution. From not being in control of when I am online, invisible and offline (signing in to Windows DOES NOT MEAN I am available for chatting) to my chatting account being different from the one I use to sign in (I have 3 live accounts) there are so many things wrong with this solution. I finally found the "Accounts" setting for the Messaging application, but I cannot add more live accounts, nor can I change the one I signed in with.
I installed the currently available version of Live Messenger from the Web, and then I can just use it like I did before. So disable Messaging live tile, install Live Essentials, and good go go
Task Manager
You have to love the new Task Manager. It really works well for the power user, shows lots of details and lets you dig deep into how the system is working, what is running. It shows per application network usage, lets you manage the system services. Wonderful improvement.
Application switching
I can now switch to the desktop right from the Start screen. No tricks needed (like last time). By default there is a tile in the start screen that switches to the desktop. You can also switch there if you have multiple monitors by clicking on the secondary display. Because the start screen is only visible on the primary display.
And yes, Alt-Tab now switches between ALL open applications, both Metro-style and traditional applications. Also very good.
And you can move your Start screen and full screen metro apps to another display as well.
Corner menus
When you move your mouse to one of the corners, something will happen. Upper left corner allows you to switch application, and even to list the metro-style apps for easy switching. Lower left corner will allow you to go to the start screen or the regular desktop. Lower and upper right corner will show the charms bar.
They are good until you have multiple monitors connected. When you have multiple monitors, the secondary monitor will be either left or right side of the first. These corner shortcuts will thus fall onto the BORDER between two monitors. So it actually becomes an aiming excercise with your mouse to aim for the primary display's corner that now lies BETWEEN two monitors. Good luck with that!
Start menu, Searching
Windows 8 removes the start menu. Thinking about it, I hardly ever went in there. Usually I opened it with the Windows key, and searched for what I want. So it is not a big loss for me.
Windows 8 changes searching in a way, that the default search does not always find what I am looking for. You know the sound dialog, where you can pick your playback and recording devices? I usually typed sound into the Start menu to find it. It was the first or second pick.
If you do this in Windows 8, it will not find it. Because this dialog is categorized as a setting, not as an application. You see, there are three searches now: Applications, Settings and Files. You have to know in advance, what you are searching for, because only those results are displayed. You can change that, but it requires extra clicks or keys. Anyhow, remembering the three quick keys for these makes search usable (Applications: Win+q, Settings: Win+w, Files: Win+f)
Hard to find options
Every wanted to shut down Windows 8 - did you succeed without using Bing or Google to look up how to do it? Intuitiveness is very important in user interfaces. It is something that should be very very important in every user interface. People are lazy reading manuals, and they will get frustrated when something does not work they way they expect it. Shutting down or restarting Windows 8 is one of these things.
Again, my analogy - in Windows 7 many options feel very natural, and you really do not need much learning to use it. You might say I am accustomed to the way Windows works, but I also use Mac OS. I did not have trouble finding things there either, and I certainly did not have to use Google to figure out how to turn off the computer.
Task bar in desktop mode
A small complaint, but the task bar is now visible on both monitors in multi-monitor setup. Hopefully there is an option in there somewhere, that can make it appear only on the primary monitor. Not a big deal though.
Further notes
Just some random observations.
My mice have back and forward buttons (one is a Microsoft mouse, the other is a Razer). They usually work even without any drivers. However, Windows 8 does not respect these buttons, for example, in the Store, when there is clearly a back button on the user interface. Pressing mouse buttons does not move you anywhere.
Summary
Compared to the Developer Preview, there are some advancements in usability. The start menu gone, but you can launch your applications and with time, find your way around Windows again. I just do not understand why Microsoft fails to see how the Metro-style operation is not something that seamlessly works with the desktop.
I will not say that you could not renew the way Windows works. By all means, there are probably always improvements to make. But a mouse and a touchscreen with a finger will require different ways to operate things, and I simply do not believe that you can get away by trying to use the same solution to both scenarios.
Overall, apart from some usability complaints (which are explored in the rest of this article), I could probably imagine working in Windows 8. The UI still feels strange and a little bit clumsy. Probably this is because I do not think the Metro style UI and applications are in harmony with the desktop paradigm. Anyhow, there are improvements compared to the developer preview, but I still find it is not as smooth of a user experience as Windows 7 is today.
Update: If you are a heavy keyboard user, here is a list of keyboard shortcuts that come in very handy with Windows 8. Very useful! Windows 8 keyboard shortcuts.
If Windows 8 would be released today, I do not think I would want to upgrade. The mix of Metro-style and regular applications is not as smooth as I would want it to be. Frankly, I do not think this new Metro-style thing belongs to the desktop, and I for one do not see as an improvement.
Features of Windows 8
I will not try to introduce the features of Windows 8, because there are tons of articles with pictures. Please refer to Engadget for one such article. And I really like the punch line for that article:
We really liked Windows 7 when it launched. It felt like a big step forward in the short time that had passed since Vista. Now, as we creep closer to a likely release near the end of this year, we can't shake a sense of doubt. Windows 8 still feels like two very different operating systems trying to be one. The potential is hugely alluring -- a single OS to rule both the tablet and the desktop -- and with each subsequent version we keep hoping this will be the one that ties it all together. Sadly, as of the Consumer Preview, we're still seeing a lot of loose threads.
As it stands, Windows 8 is a considerably better tablet operating system than any previous version has managed to be. However, it's still a clumsier desktop OS than Windows 7. That's a problem Microsoft must fix before release. (from Engadget)
The Metro experience
Anyone reading about or trying out Windows 8 will probably know what Metro-style applications are. They can be started from the new Start screen, they run full screen, and cannot be removed from being full screen (other than using snapping). Running applications full screen seems natural for certain solutions. For web browsers, it is a natural way to operate. For some other applications, it just isn't.
Sidenote: Why kill windows in Windows? If everything is full screen, there are no windows anymore. Hmm?
Full screen
For some applications, running them full screen seems outright ridiculous. For example, take the default calendar application on a 24" screen. On a 10" tablet those might look nice, but it just does not work on big screens. Messaging application - I never-ever run chat/messenger in full screen. I like having it floating around above/near other windows. I can do work in my main window and follow the chat at the same time.
I have created some tools and small programs that run as quite small Windows. There would be no sense in making an application full screen, that requires 2 or 3 textboxes for input. I simply cannot imagine how such a tool could be reimagined into being a Metro-style application. Of course if I wanted to create a smartphone or tablet version, sure, no problem. But putting 2 textboxes and a small image on a 24" screen? Or 30" for that matter. I just do not think this is a universally good idea. (I am about to explore Windows 8 application development further, so I might revise this paragraph in the future)
You cannot switch Metro-style apps to run in a Window. Going back to the calendar application: I usually have a webpage, email, chat or something else opened to which the calendar would relate. Having windows side by side makes it much easier than having to alt-tab all the time. I have a high resolution 24" monitor at home, and two large displays side-by-side at work. I have them not because I enjoy looking at full screen things. I have big screen real-estate to actually use it for MULTIPLE WINDOWS at the same time. Forcing new apps (Metro-style) to run full-screen, it is a waste of screen real estate.
Snapping applications partially solves this problem. You can use Windows+. and Windows+Shift+. keys to snap application windows. You can even snap a Metro style app (the calendar) and then the remaining space can be used by the desktop. And mind you, snapping is not something you would figure out for yourself - I certainly didn't
Normally I can remove apps from full screen, and very easily put them side by side. Just press "Windows+left" and "Windows+right" and you can arrange your apps very fast in Windows 7. Or use the mouse and drag the title bar to the left side, top side or right side. Very very easy and very very fast. I was actually very impressed, how much thought went into the usability of Windows 7. With Windows 8, I do not find that Metro-style apps improved this in any way, even considering snapping.
I really do not care how much Microsoft tries to justify this new user interface. When Windows 7 came out, you could feel the improvement. With Windows 8, you can feel the setback in user experience.
When you look at the changes from the tablet perspective, it of course makes all sense. I can understand ALL of these choices and behaviors when using a mobile phone or a tablet. There I would even expect everything to work like Metro does. And I don't doubt that Windows 8 tablets will be very nice to use. BUT the desktop is different, and using the same paradigm for the desktop will hinder performance and efficiency.
Messenger
I seem to be signed in to messenger as soon as I sign in to Windows (because I used a Live account to sign in). Where do I began to list all my problems with this solution. From not being in control of when I am online, invisible and offline (signing in to Windows DOES NOT MEAN I am available for chatting) to my chatting account being different from the one I use to sign in (I have 3 live accounts) there are so many things wrong with this solution. I finally found the "Accounts" setting for the Messaging application, but I cannot add more live accounts, nor can I change the one I signed in with.
I installed the currently available version of Live Messenger from the Web, and then I can just use it like I did before. So disable Messaging live tile, install Live Essentials, and good go go
Task Manager
You have to love the new Task Manager. It really works well for the power user, shows lots of details and lets you dig deep into how the system is working, what is running. It shows per application network usage, lets you manage the system services. Wonderful improvement.
Application switching
I can now switch to the desktop right from the Start screen. No tricks needed (like last time). By default there is a tile in the start screen that switches to the desktop. You can also switch there if you have multiple monitors by clicking on the secondary display. Because the start screen is only visible on the primary display.
And yes, Alt-Tab now switches between ALL open applications, both Metro-style and traditional applications. Also very good.
And you can move your Start screen and full screen metro apps to another display as well.
Corner menus
When you move your mouse to one of the corners, something will happen. Upper left corner allows you to switch application, and even to list the metro-style apps for easy switching. Lower left corner will allow you to go to the start screen or the regular desktop. Lower and upper right corner will show the charms bar.
They are good until you have multiple monitors connected. When you have multiple monitors, the secondary monitor will be either left or right side of the first. These corner shortcuts will thus fall onto the BORDER between two monitors. So it actually becomes an aiming excercise with your mouse to aim for the primary display's corner that now lies BETWEEN two monitors. Good luck with that!
Start menu, Searching
Windows 8 removes the start menu. Thinking about it, I hardly ever went in there. Usually I opened it with the Windows key, and searched for what I want. So it is not a big loss for me.
Windows 8 changes searching in a way, that the default search does not always find what I am looking for. You know the sound dialog, where you can pick your playback and recording devices? I usually typed sound into the Start menu to find it. It was the first or second pick.
If you do this in Windows 8, it will not find it. Because this dialog is categorized as a setting, not as an application. You see, there are three searches now: Applications, Settings and Files. You have to know in advance, what you are searching for, because only those results are displayed. You can change that, but it requires extra clicks or keys. Anyhow, remembering the three quick keys for these makes search usable (Applications: Win+q, Settings: Win+w, Files: Win+f)
Hard to find options
Every wanted to shut down Windows 8 - did you succeed without using Bing or Google to look up how to do it? Intuitiveness is very important in user interfaces. It is something that should be very very important in every user interface. People are lazy reading manuals, and they will get frustrated when something does not work they way they expect it. Shutting down or restarting Windows 8 is one of these things.
Again, my analogy - in Windows 7 many options feel very natural, and you really do not need much learning to use it. You might say I am accustomed to the way Windows works, but I also use Mac OS. I did not have trouble finding things there either, and I certainly did not have to use Google to figure out how to turn off the computer.
Task bar in desktop mode
A small complaint, but the task bar is now visible on both monitors in multi-monitor setup. Hopefully there is an option in there somewhere, that can make it appear only on the primary monitor. Not a big deal though.
Further notes
Just some random observations.
My mice have back and forward buttons (one is a Microsoft mouse, the other is a Razer). They usually work even without any drivers. However, Windows 8 does not respect these buttons, for example, in the Store, when there is clearly a back button on the user interface. Pressing mouse buttons does not move you anywhere.
Summary
Compared to the Developer Preview, there are some advancements in usability. The start menu gone, but you can launch your applications and with time, find your way around Windows again. I just do not understand why Microsoft fails to see how the Metro-style operation is not something that seamlessly works with the desktop.
I will not say that you could not renew the way Windows works. By all means, there are probably always improvements to make. But a mouse and a touchscreen with a finger will require different ways to operate things, and I simply do not believe that you can get away by trying to use the same solution to both scenarios.
Nokia and Windows Phone
(Mobile)
This article will discuss my opinion on Nokia and Windows Phone. If someone reading it notices that I got my facts wrong, please do correct me
Updates to this article appended to the end.
A Hungarian proverb
There is a Hungarian proverb that translates roughly like so: one of my eyes if crying, the other is laughing. It refers to some kind of thing which is both good and bad for a person. And that is what I feel when I think about Nokia and Windows Phone adoption.
Crying? I am just not reassured, that Windows Phone will be so great for Nokia. This feeling might be caused by my previous clash with the Windows Mobile platform (pre-Windows Phone 7), that you can read about here. This obviously does not apply to Windows Phone 7 (or at least not entirely), but I can't help my bad feelings. There are some features of Symbian (we will get into this later) that are missing from WP7 (and also from iPhone or Android).
I will admit I never used a WP7 phone for more than a short time in a shop or lending it from a friend to try things out. I have also done some research (reading about it, reading phones manuals). My opinion is based on these facts. But I still have that same feeling which I had with Windows Mobile previously: that you are not in control. The phone (or Microsoft) does certain decisions on how you should use your mobile phone, and the restraints just make me more angry I know many would recommend Android to me based on these facts. Maybe. (I will also admit at this point, that some features I critisize here might already be available on Android, or might be available via separate apps, etc. My Android information is dated )
Laughing? So why would I be happy about this move of Nokia? I have been developing in .NET since 2004, both professionally and for a hobby. I really really like the platform, and Microsoft's strategy to have the same programming APIs across a wide range of devices is just really tempting. I do not need new tools or new skills for that matter (as I have programmed Silverlight for example), I can just hop in and start coding Windows Phone 7 (I have written some apps in the emulator already). And I can soon run it on a Nokia, which sounds pretty interesting.
Profiles and groups - are features dying?
Wow, Windows Phone 7.5 Mango now supports grouping your users. That is awesome. Nokia phones did it 10 years ago, and in my opinion better too. In Mango groups serve only to be able to view a group of users together (their socials updates, etc) and to communicate with them. Granted, groups might not be integrated into the social aspects of newer Nokia phones, but what about basic "PHONE" settings? Like setting a ringtone for a group? Like selecting which groups will alert you (ring) in a given profile?
... Oh wait ...
Windows Phone does not have profiles? Well it sure seems so. Set a ringtone. Check. You can have it on silent. Check. You can have it on vibrate. Check. You can't use vibrate and silent at the same time. What?
If I want to have silent available, I have to TURN OFF Vibrate in settings? To have vibrate on again, need to set settings again. Otherwise, the volume down button will put you into either SILENT or VIBRATE. Frankly, that seems just stupid. And I wonder if anyone who wrote this part of Windows Phone has ever seen how Nokia does profiles? (yes, I doubt the person has seen that feature in action).
My only hope is, that Nokia will port these missing features from Symbian into their Windows Phone offering. That would give them a huge edge over other Windows Phone manufacturers too - at least for some time, as they have to contribute those changes back into WP7. But that would just make WP7 better. However, I am afraid this porting over features will not happen - but who knows?
And do not even get me started on things like "who needs profiles?" and "people do not use their phones to make phone calls anymore", because those things are just stupid.
To sum up this section, I think that Symbian offered a lot of control over your settings and a lot of settings to choose from and customize, how your phone works. Loosing these things in the transition to Windows Phone would be unacceptable to me, but we will have to see what the good guys in Nokia think about this. Granted, the settings were not logically organized and often hard to find, but they were still there. I dislike the "we know better than the user" design philosophy, which is visible in both Windows and Windows Phone.
If features from Symbian end up missing from the WP7 Nokias, that is just a downgrade, plain and simple.
Why Software Sucks?
But why does Nokias software suck?
As a side note, there is an excellent book called Why Software Sucks? I do recommend anyone involved in software development to read it
But back to Nokia. Why doesn't Symbian work so well? I would think this is because the systems core is ancient (at least by computing standards) and building on a not so solid foundation is just not that easy. But Nokia has some capable people, and you can see the results by looking at the N9 and MeeGo. You have to wonder where Nokia kept that software talent all these years, because that thing really works well. Also note, that my criticism is only for the software side of Nokia phones. Their hardware is really something to admire, especially recently. I just wished their software would been on par with the hardware they make.
Nokia handled some things concerning their software badly. The software development speed, update cycles and software philosophy left a lot to be desired. I have seen it many times, that old phones just do not get all the nice updates new phones have gotten. This is now changing. In the Elop world, Symbian^3 devices get all the updates that newer phones come out with. Sweet. I hated that you need a new phone every 6 months to get the new features. It seems now all manufacturers are updating - more or less - old models with new software. It might seem like loosing money, but it is building up customer satisfaction. Besides, people often get locked into contracts, and cannot upgrade their phone so often. By releasing new software for new phones only, you just make those people angry.
The software updates with Nokia are coming SO SLOOOOW. There are bugs to fix, applications to update, and the updates are just coming very slow. When Symbian Anna finally arrived in the summer, it arrived one operator, one phonemodel at a time, and many months later than promised. I have been reading Nokia forums and found out, that they need to test every variant of the software. Presumably by hand. And there are variants for different color versions of the phone. Yes. You read that correct.
I see Nokia wasting money here. Money and time. And by that, customer satisfaction.
Building packages for all phone models, variants and operators SHOULD BE automatic. These variants then SHOULD BE tested automatically. I would have expected that a company like Nokia can press a button at night, and in the morning see if everything went ok, without a single person being involved in the software testing process. All packages should be built and packaged automatically and then tested, also automatically. Judging from the speed they do things, it is just not how things progress. (The reality of testing mobile phone software might be more complex than what I described above, but still it should be more easier than it seems to be now.)
Because the update process is such a big pain, of course they will not want to do it too often, so what we get is updates coming really slowly.
A new software platform again?
It was in the news that Nokia is making a cheap, simple, Linux based platform that it wants to put on the cheaper phones. And you know, this has me wondering. Why oh why do we need ANOTHER PLATFORM? You already have MeeGo, don't you? Instead of thinking about adopting that for cheaper phones, you go ahead and start developing something new again? Excuse my language, but that does seem like bloody stupid to me.
The whole technology industry is about product lines.
Take hardware. Intel processors. If in the production some processors do not qualify high enough, they have some features disabled and labeled cheaper/lower end models. Intel is even selling processors where you can unlock faster speeds AFTER you bought it by paying for an unlock code. I do think that practice is disgusting, but it does point out one key thing. They do not design and manufacture as many processor types are they sell. They simply lock out features of high end models to make them low end. Same goes for graphics cores.
Take software. You create a nice little program with a lot of feautes. Name it the Enterprise version. You lock some features down, that is your Professional version. You take away more features, Home version. And heck, make even a free Express version too. You can see it all over the place. Microsoft does it with Windows, and most independent software vendors do it too. It is much easier to lock certain things down than to create entire new versions. No one really thinks Windows Home Premium is a complete rewrite of Windows Ultimate, with certain things left out?
Why does not a company like Nokia go this way? Why make something new, when looking at the N9, you already have a very mature and good operating system. Use that. Lock things down, but use that.
The same argument goes for the now dying S40 platform. I never quite understood why you need a different platform.
There might be certain hardware restrictions to work with - like cheaper models not getting so strong hardware - but I will argue that having a single codebase is still cheaper than multiple code bases. Cheaper to make, cheaper to maintain and test.
A mistake?
I will go as far as to saying Nokia's decision to adopt Windows Phone seems like a mistake to me. Looking what Nokia has available today, it is very clear that these things have been in the making when they decided to go the Windows Phone 7 way. These offerings available now seem to me that these might have been the solution to their problem, and not adopting Windows Phone 7. Frankly, it makes you wonder about all the theories concerning why Elop chose his former employer's mobile platform.
I will explain.
Nokia now has MeeGo ready. N9 has appeared in stores. If there would be no Windows Phone 7 in the making, MeeGo devices could now appear one after the other. Read the reviews of the N9 over the internet. Praises. The user interface is compared to the iPhone, the user experience is said to be great. And some people are sad because it is not available in their country. The reaction would have been the same without Windows Phone 7. In fact, it would be even much much better, because everyone could look forward to improvements to the MeeGo platform and new devices. Not today.
On the development side, S^3 and MeeGo can be both targeted using the same development tools (Nokia Qt). So developers really do not need a lot of effort to transition. The development tools BTW also came a HUUUGE way forward, they are easy to install, maintain and use. Ovi Store is available today with lots of apps.
So right there, you already have a complete mobile platform and ecosystem available. But Nokia had to destroy it all. I do not believe for a second that four mobile ecosystems would have been too much.
Of course MeeGo cannot work in its current form, because both platforms (S^3 and MeeGo) are essentially dead ends. Current developers will work with them, but I really doubt anyone new will invest lots of time and money in training new developers for these platforms.
So, instead of the already ready and fine working MeeGo, we get Windows Phone 7 Nokia phones, that will maybe work OK, but will certainly need time to fine tune and get used to. I cannot even imagine when Nokia will be able to make a phone like the N8, with all those multimedia features: photo picture quality, video editing features, etc. As far as I know, WP7 does not have video editing?
Frankly I did not see the driving reason behind WP7 adoption back in the beginning of the year, and I see it even less now. I think Nokia threw away a tremendous opportunity by choosing Windows Phone 7, and I really doubt they can climb up to top spot again. Very sad actually.
I can really understand the comments by former Nokia chief Anssi Vanjoki who also said Windows Phone 7 was not needed by Nokia. If you understand Finnish, read his comments here.
Update 22.10.2011: Engadget just reviewed the N9, coming to similar conclusions: it is just a shame Nokia killed of MeeGo.
Updates to this article appended to the end.
A Hungarian proverb
There is a Hungarian proverb that translates roughly like so: one of my eyes if crying, the other is laughing. It refers to some kind of thing which is both good and bad for a person. And that is what I feel when I think about Nokia and Windows Phone adoption.
Crying? I am just not reassured, that Windows Phone will be so great for Nokia. This feeling might be caused by my previous clash with the Windows Mobile platform (pre-Windows Phone 7), that you can read about here. This obviously does not apply to Windows Phone 7 (or at least not entirely), but I can't help my bad feelings. There are some features of Symbian (we will get into this later) that are missing from WP7 (and also from iPhone or Android).
I will admit I never used a WP7 phone for more than a short time in a shop or lending it from a friend to try things out. I have also done some research (reading about it, reading phones manuals). My opinion is based on these facts. But I still have that same feeling which I had with Windows Mobile previously: that you are not in control. The phone (or Microsoft) does certain decisions on how you should use your mobile phone, and the restraints just make me more angry
I know many would recommend Android to me based on these facts. Maybe. (I will also admit at this point, that some features I critisize here might already be available on Android, or might be available via separate apps, etc. My Android information is dated )
Laughing? So why would I be happy about this move of Nokia? I have been developing in .NET since 2004, both professionally and for a hobby. I really really like the platform, and Microsoft's strategy to have the same programming APIs across a wide range of devices is just really tempting. I do not need new tools or new skills for that matter (as I have programmed Silverlight for example), I can just hop in and start coding Windows Phone 7 (I have written some apps in the emulator already). And I can soon run it on a Nokia, which sounds pretty interesting.
Profiles and groups - are features dying?
Wow, Windows Phone 7.5 Mango now supports grouping your users. That is awesome. Nokia phones did it 10 years ago, and in my opinion better too. In Mango groups serve only to be able to view a group of users together (their socials updates, etc) and to communicate with them. Granted, groups might not be integrated into the social aspects of newer Nokia phones, but what about basic "PHONE" settings? Like setting a ringtone for a group? Like selecting which groups will alert you (ring) in a given profile?
... Oh wait ...
Windows Phone does not have profiles? Well it sure seems so. Set a ringtone. Check. You can have it on silent. Check. You can have it on vibrate. Check. You can't use vibrate and silent at the same time. What?
If I want to have silent available, I have to TURN OFF Vibrate in settings? To have vibrate on again, need to set settings again. Otherwise, the volume down button will put you into either SILENT or VIBRATE. Frankly, that seems just stupid. And I wonder if anyone who wrote this part of Windows Phone has ever seen how Nokia does profiles? (yes, I doubt the person has seen that feature in action).
My only hope is, that Nokia will port these missing features from Symbian into their Windows Phone offering. That would give them a huge edge over other Windows Phone manufacturers too - at least for some time, as they have to contribute those changes back into WP7. But that would just make WP7 better. However, I am afraid this porting over features will not happen - but who knows?
And do not even get me started on things like "who needs profiles?" and "people do not use their phones to make phone calls anymore", because those things are just stupid.
To sum up this section, I think that Symbian offered a lot of control over your settings and a lot of settings to choose from and customize, how your phone works. Loosing these things in the transition to Windows Phone would be unacceptable to me, but we will have to see what the good guys in Nokia think about this. Granted, the settings were not logically organized and often hard to find, but they were still there. I dislike the "we know better than the user" design philosophy, which is visible in both Windows and Windows Phone.
If features from Symbian end up missing from the WP7 Nokias, that is just a downgrade, plain and simple.
Why Software Sucks?
But why does Nokias software suck?
As a side note, there is an excellent book called Why Software Sucks? I do recommend anyone involved in software development to read it
But back to Nokia. Why doesn't Symbian work so well? I would think this is because the systems core is ancient (at least by computing standards) and building on a not so solid foundation is just not that easy. But Nokia has some capable people, and you can see the results by looking at the N9 and MeeGo. You have to wonder where Nokia kept that software talent all these years, because that thing really works well. Also note, that my criticism is only for the software side of Nokia phones. Their hardware is really something to admire, especially recently. I just wished their software would been on par with the hardware they make.
Nokia handled some things concerning their software badly. The software development speed, update cycles and software philosophy left a lot to be desired. I have seen it many times, that old phones just do not get all the nice updates new phones have gotten. This is now changing. In the Elop world, Symbian^3 devices get all the updates that newer phones come out with. Sweet. I hated that you need a new phone every 6 months to get the new features. It seems now all manufacturers are updating - more or less - old models with new software. It might seem like loosing money, but it is building up customer satisfaction. Besides, people often get locked into contracts, and cannot upgrade their phone so often. By releasing new software for new phones only, you just make those people angry.
The software updates with Nokia are coming SO SLOOOOW. There are bugs to fix, applications to update, and the updates are just coming very slow. When Symbian Anna finally arrived in the summer, it arrived one operator, one phonemodel at a time, and many months later than promised. I have been reading Nokia forums and found out, that they need to test every variant of the software. Presumably by hand. And there are variants for different color versions of the phone. Yes. You read that correct.
I see Nokia wasting money here. Money and time. And by that, customer satisfaction.
Building packages for all phone models, variants and operators SHOULD BE automatic. These variants then SHOULD BE tested automatically. I would have expected that a company like Nokia can press a button at night, and in the morning see if everything went ok, without a single person being involved in the software testing process. All packages should be built and packaged automatically and then tested, also automatically. Judging from the speed they do things, it is just not how things progress. (The reality of testing mobile phone software might be more complex than what I described above, but still it should be more easier than it seems to be now.)
Because the update process is such a big pain, of course they will not want to do it too often, so what we get is updates coming really slowly.
A new software platform again?
It was in the news that Nokia is making a cheap, simple, Linux based platform that it wants to put on the cheaper phones. And you know, this has me wondering. Why oh why do we need ANOTHER PLATFORM? You already have MeeGo, don't you? Instead of thinking about adopting that for cheaper phones, you go ahead and start developing something new again? Excuse my language, but that does seem like bloody stupid to me.
The whole technology industry is about product lines.
Take hardware. Intel processors. If in the production some processors do not qualify high enough, they have some features disabled and labeled cheaper/lower end models. Intel is even selling processors where you can unlock faster speeds AFTER you bought it by paying for an unlock code. I do think that practice is disgusting, but it does point out one key thing. They do not design and manufacture as many processor types are they sell. They simply lock out features of high end models to make them low end. Same goes for graphics cores.
Take software. You create a nice little program with a lot of feautes. Name it the Enterprise version. You lock some features down, that is your Professional version. You take away more features, Home version. And heck, make even a free Express version too. You can see it all over the place. Microsoft does it with Windows, and most independent software vendors do it too. It is much easier to lock certain things down than to create entire new versions. No one really thinks Windows Home Premium is a complete rewrite of Windows Ultimate, with certain things left out?
Why does not a company like Nokia go this way? Why make something new, when looking at the N9, you already have a very mature and good operating system. Use that. Lock things down, but use that.
The same argument goes for the now dying S40 platform. I never quite understood why you need a different platform.
There might be certain hardware restrictions to work with - like cheaper models not getting so strong hardware - but I will argue that having a single codebase is still cheaper than multiple code bases. Cheaper to make, cheaper to maintain and test.
A mistake?
I will go as far as to saying Nokia's decision to adopt Windows Phone seems like a mistake to me. Looking what Nokia has available today, it is very clear that these things have been in the making when they decided to go the Windows Phone 7 way. These offerings available now seem to me that these might have been the solution to their problem, and not adopting Windows Phone 7. Frankly, it makes you wonder about all the theories concerning why Elop chose his former employer's mobile platform.
I will explain.
Nokia now has MeeGo ready. N9 has appeared in stores. If there would be no Windows Phone 7 in the making, MeeGo devices could now appear one after the other. Read the reviews of the N9 over the internet. Praises. The user interface is compared to the iPhone, the user experience is said to be great. And some people are sad because it is not available in their country. The reaction would have been the same without Windows Phone 7. In fact, it would be even much much better, because everyone could look forward to improvements to the MeeGo platform and new devices. Not today.
On the development side, S^3 and MeeGo can be both targeted using the same development tools (Nokia Qt). So developers really do not need a lot of effort to transition. The development tools BTW also came a HUUUGE way forward, they are easy to install, maintain and use. Ovi Store is available today with lots of apps.
So right there, you already have a complete mobile platform and ecosystem available. But Nokia had to destroy it all. I do not believe for a second that four mobile ecosystems would have been too much.
Of course MeeGo cannot work in its current form, because both platforms (S^3 and MeeGo) are essentially dead ends. Current developers will work with them, but I really doubt anyone new will invest lots of time and money in training new developers for these platforms.
So, instead of the already ready and fine working MeeGo, we get Windows Phone 7 Nokia phones, that will maybe work OK, but will certainly need time to fine tune and get used to. I cannot even imagine when Nokia will be able to make a phone like the N8, with all those multimedia features: photo picture quality, video editing features, etc. As far as I know, WP7 does not have video editing?
Frankly I did not see the driving reason behind WP7 adoption back in the beginning of the year, and I see it even less now. I think Nokia threw away a tremendous opportunity by choosing Windows Phone 7, and I really doubt they can climb up to top spot again. Very sad actually.
I can really understand the comments by former Nokia chief Anssi Vanjoki who also said Windows Phone 7 was not needed by Nokia. If you understand Finnish, read his comments here.
Update 22.10.2011: Engadget just reviewed the N9, coming to similar conclusions: it is just a shame Nokia killed of MeeGo.
Windows 8 first impressions
(Windows)
I gave Windows 8 a try the other day. While I have not clocked a lot of usage time with it, here are my first impressions.
Login
I liked that I can use my Live id to login, and that it will synchronize my user settings around computers. I like it expecially since at home I have multiple computers, but no domain. With an Active Directory domain you can accomplish about the same (roaming profiles), but until now it was not possible with normals accounts. At least without some custom applications. At least Chrome synchs browser settings (even themes and extensions), so in our web centric world today, that makes it really easier to use many computers. And with W8, this might be even better
I noticed that it does create a local user account even when using Live. It somehow links this local account to your Live ID. I would love to know more about how this integration technically works. Background information anyone?
Start Screen
There is a new Start Screen in Windows 8. There are actually several articles from Microsoft that address the Start Screen, one of the latest being:
Designing the Start Screen
The article explains that study shows, people are moving away from the start menu. People are pinning programs directly to the task bar and launching them from there. This is very true, and I can say that I have been moving in this direction myself. The most frequently used programs are on the task bar. The less frequently used programs are pinned to my Start Menu. And the rest is possibly listed in the recently used portion of my Start menu. And if I still do not find something, I just use Search (press Windows button on keyboard and start typing). It is very very rare for me to venture into the Start Menus. So probably this research does tell the truth and people are indeed using the Start Menu less often.
Ok, so Microsoft now replaces the Start Menu with the Start Screen, which is a sort of dashboard, will all applications on it, live tiles with information, etc.
Continuing the previous use case of rarely used programs: I do have to point out however, that on those rare occasions, it is very easy to find the program I am looking for in the Start Menu. Because if I need something Visual Studio related, I look in that folder of Start Menu. If I need something Office related, I look in Microsoft Office folder. The new Start Screen just lists everything it finds in the Start Menu. So now with this new Screen on the rare occasion I need something thats name I do not remember (cannot search), it actually makes it HARDER to find it, because everything is in one big messy pile of icons. Yes, I can customize the Start Screen, but we are talking about rarely used stuff, why would I want to customize that, when it is already in neat folders in my Start Menu?
It covers the whole screen
But the new Start Screen in Windows 8 has several more disadvantages. As others have pointed out, it COVERS THE WHOLE SCREEN. So I do not see at a glance, what I have already running, and what is not running on the computer. As soon as I launch into the Start Screen, the task bar is hidden. This full screen operation mode does not suit desktops in my opinion. It might suit tablets, but definitely not the desktop.
The article also mentions, that you can still use Win+1 or Win+2 to launch programs from the task bar. That is great, except wait, I cannot see my task bar from the Start Screen. And when I cannot see it, I also cannot use those shortcuts! (well until I visit the desktop at least once, which is not trivial task either)
To make matters worse, not even the mouse works until you CLICK on the Start Screen, which is already visible (BUG?). You cannot close the Start Screen and see your desktop until you start something. After that, even if you exit that, you can use ESCAPE to go back to Desktop or press the Windows button. (again, BUG?)
Notifications
Next, Microsoft says that the new Start Screen brings together notifications, which were poorly implemented in the taskbar popups and notification tray icons in previous versions of Windows. The start screen connects you to apps, it shows news, RSS feeds, weather, and so on.
Now this is very nice, but guess what, when I am using my computer, I do not spend time on the Start Screen. I spend time in Visual Studio, in Word, in Photoshop, in my browser or in Outlook (or another application). There might be more application windows visible on my desktop side by side, maybe even on multiple monitors. Those notification that popup on the screen or from the task bar / notification tray are a way to get my attention. I am not going to check back to the Start Screen to see if anything new has happened. I am not going to sit around the Start Screen waiting for emails or facebook messages. I will be spending time in Visual Studio or in Word or in whatever application I am using, and when that notification icon appears in the task bar icon of Outlook I am going to switch over there and check the mail. And I am not going to go through a Start Screen.
A Start Screen is a good point to START things, but during work or operation, it is not something I want to use, at least not in its current form. Now when it pops up fullscreen, it just obscures everything and gets in the way of getting things done.
While I criticize how it works now, I do see the need and use of a centralized hub of information, where you can check out notifications all at once, check if new news came in, check if there are new bugs, check if there are new messages, chat messages, emails, status updates on Facebook, etc. But I would want that to be less disruptive than launching into the Start Screen. Besides, if I need to switch over to the Start Screen to check for notifications, it just beats the whole purpose of notifications in the first place.
The keyboard experience, searching
It is not all bad - but I will say it definitely needs fine tuning for the desktop use. And it needs tuning to be used with the keyboard.
I am a big fan of using Windows with the keyboard, and I was very satisfied by how Windows 7 improved the keyboard only experience of using the operating system. With the Windows 7 Start Menu, I can push the Start button on the keyboard, start typing (into search), use arrows to select from list, and launch. Often the first result was good enough, so I could just hit enter to start the application. The new Start Screen works like this if what you want is an application. It also displays more results at once. But in certain situations the experience is worse than in Windows 7. Not to mention that the Start Screen covers all apps and the taskbar. Not good.
What I particularly did not like about search in Windows 8, is that it separates Apps, Settings and Files. It just displays results from Apps as a default (in Windows 7, all results were displayed at once). What does this mean? On Windows 7, when I type in “add remove” into Windows 7 search box, the first result is “Add or remove programs”. I can hit enter and start working. In Windows 8 the result is bogus ("Add or remove help content", obviously some kind of App), until I remember to switch over to Settings under the search box. Which is not even trivial using the keyboard.
As mentioned before, Start Screen covers the screen, so naturally so do search results. As the article says, the start menu could not scale to fit all the search results: this is true. In this respect the new Start Screen is superior, because in can display more entries.
I would stress that the task bar needs to be visible all the time. When Windows starts, by all means, show the Start Screen. Maybe even full screen. But almost always I would see a better option of the Start Screen being a transparent overlay (check out OS X Dashboard) over the desktop, excluding the taskbar (which should be always visible).
Summary
This is of course just a developer preview, and I hope Microsoft can fine tune the new Start Screen, especially for desktop users. I would say the keyboard (and mouse) user experience needs a lot of work, because in its current form it is more disruptive than helpful.
When Windows Vista came out, and after that, Windows 7, the new Start Menu and its features struck me as something that increased my productivity, especially with the keyboard. I just do not feel the new Start Screen does the same. I feel it hinders me. I realize sometimes you need drastic changes to get new things going, but I just do no feel the Start Screen in its current form satisfies this goal. It might be something really cool for tablets, but desktop users are left behind in its current version.
Eagerly waiting more Windows 8 previews And I still need to check out development tools for Windows 8.
Login
I liked that I can use my Live id to login, and that it will synchronize my user settings around computers. I like it expecially since at home I have multiple computers, but no domain. With an Active Directory domain you can accomplish about the same (roaming profiles), but until now it was not possible with normals accounts. At least without some custom applications. At least Chrome synchs browser settings (even themes and extensions), so in our web centric world today, that makes it really easier to use many computers. And with W8, this might be even better
I noticed that it does create a local user account even when using Live. It somehow links this local account to your Live ID. I would love to know more about how this integration technically works. Background information anyone?
Start Screen
There is a new Start Screen in Windows 8. There are actually several articles from Microsoft that address the Start Screen, one of the latest being:
Designing the Start Screen
The article explains that study shows, people are moving away from the start menu. People are pinning programs directly to the task bar and launching them from there. This is very true, and I can say that I have been moving in this direction myself. The most frequently used programs are on the task bar. The less frequently used programs are pinned to my Start Menu. And the rest is possibly listed in the recently used portion of my Start menu. And if I still do not find something, I just use Search (press Windows button on keyboard and start typing). It is very very rare for me to venture into the Start Menus. So probably this research does tell the truth and people are indeed using the Start Menu less often.
Ok, so Microsoft now replaces the Start Menu with the Start Screen, which is a sort of dashboard, will all applications on it, live tiles with information, etc.
Continuing the previous use case of rarely used programs: I do have to point out however, that on those rare occasions, it is very easy to find the program I am looking for in the Start Menu. Because if I need something Visual Studio related, I look in that folder of Start Menu. If I need something Office related, I look in Microsoft Office folder. The new Start Screen just lists everything it finds in the Start Menu. So now with this new Screen on the rare occasion I need something thats name I do not remember (cannot search), it actually makes it HARDER to find it, because everything is in one big messy pile of icons. Yes, I can customize the Start Screen, but we are talking about rarely used stuff, why would I want to customize that, when it is already in neat folders in my Start Menu?
It covers the whole screen
But the new Start Screen in Windows 8 has several more disadvantages. As others have pointed out, it COVERS THE WHOLE SCREEN. So I do not see at a glance, what I have already running, and what is not running on the computer. As soon as I launch into the Start Screen, the task bar is hidden. This full screen operation mode does not suit desktops in my opinion. It might suit tablets, but definitely not the desktop.
The article also mentions, that you can still use Win+1 or Win+2 to launch programs from the task bar. That is great, except wait, I cannot see my task bar from the Start Screen. And when I cannot see it, I also cannot use those shortcuts! (well until I visit the desktop at least once, which is not trivial task either)
To make matters worse, not even the mouse works until you CLICK on the Start Screen, which is already visible (BUG?). You cannot close the Start Screen and see your desktop until you start something. After that, even if you exit that, you can use ESCAPE to go back to Desktop or press the Windows button. (again, BUG?)
Notifications
Next, Microsoft says that the new Start Screen brings together notifications, which were poorly implemented in the taskbar popups and notification tray icons in previous versions of Windows. The start screen connects you to apps, it shows news, RSS feeds, weather, and so on.
Now this is very nice, but guess what, when I am using my computer, I do not spend time on the Start Screen. I spend time in Visual Studio, in Word, in Photoshop, in my browser or in Outlook (or another application). There might be more application windows visible on my desktop side by side, maybe even on multiple monitors. Those notification that popup on the screen or from the task bar / notification tray are a way to get my attention. I am not going to check back to the Start Screen to see if anything new has happened. I am not going to sit around the Start Screen waiting for emails or facebook messages. I will be spending time in Visual Studio or in Word or in whatever application I am using, and when that notification icon appears in the task bar icon of Outlook I am going to switch over there and check the mail. And I am not going to go through a Start Screen.
A Start Screen is a good point to START things, but during work or operation, it is not something I want to use, at least not in its current form. Now when it pops up fullscreen, it just obscures everything and gets in the way of getting things done.
While I criticize how it works now, I do see the need and use of a centralized hub of information, where you can check out notifications all at once, check if new news came in, check if there are new bugs, check if there are new messages, chat messages, emails, status updates on Facebook, etc. But I would want that to be less disruptive than launching into the Start Screen. Besides, if I need to switch over to the Start Screen to check for notifications, it just beats the whole purpose of notifications in the first place.
The keyboard experience, searching
It is not all bad - but I will say it definitely needs fine tuning for the desktop use. And it needs tuning to be used with the keyboard.
I am a big fan of using Windows with the keyboard, and I was very satisfied by how Windows 7 improved the keyboard only experience of using the operating system. With the Windows 7 Start Menu, I can push the Start button on the keyboard, start typing (into search), use arrows to select from list, and launch. Often the first result was good enough, so I could just hit enter to start the application. The new Start Screen works like this if what you want is an application. It also displays more results at once. But in certain situations the experience is worse than in Windows 7. Not to mention that the Start Screen covers all apps and the taskbar. Not good.
What I particularly did not like about search in Windows 8, is that it separates Apps, Settings and Files. It just displays results from Apps as a default (in Windows 7, all results were displayed at once). What does this mean? On Windows 7, when I type in “add remove” into Windows 7 search box, the first result is “Add or remove programs”. I can hit enter and start working. In Windows 8 the result is bogus ("Add or remove help content", obviously some kind of App), until I remember to switch over to Settings under the search box. Which is not even trivial using the keyboard.
As mentioned before, Start Screen covers the screen, so naturally so do search results. As the article says, the start menu could not scale to fit all the search results: this is true. In this respect the new Start Screen is superior, because in can display more entries.
I would stress that the task bar needs to be visible all the time. When Windows starts, by all means, show the Start Screen. Maybe even full screen. But almost always I would see a better option of the Start Screen being a transparent overlay (check out OS X Dashboard) over the desktop, excluding the taskbar (which should be always visible).
Summary
This is of course just a developer preview, and I hope Microsoft can fine tune the new Start Screen, especially for desktop users. I would say the keyboard (and mouse) user experience needs a lot of work, because in its current form it is more disruptive than helpful.
When Windows Vista came out, and after that, Windows 7, the new Start Menu and its features struck me as something that increased my productivity, especially with the keyboard. I just do not feel the new Start Screen does the same. I feel it hinders me. I realize sometimes you need drastic changes to get new things going, but I just do no feel the Start Screen in its current form satisfies this goal. It might be something really cool for tablets, but desktop users are left behind in its current version.
Eagerly waiting more Windows 8 previews
And I still need to check out development tools for Windows 8.
Merging WSDL and XSD files
(Tips & Tricks)
Having played with BizTalk some while ago (version 2006 R2) I had an interesting problem: in certain situations BizTalk would not accept web service description files (WSDL) where the XML schema was stored in separate files (XSD). No matter how hard I tried, there was no success. I even tried the good old trick of putting the files in a webserver and trying to add them over HTTP. It would not work. (This is the way you trick the Visual Studio proxy generator when WSDL and XSD files are spread over the hard drive in different directories, but that is another story )
Finally, I ended up creating a tool I called WSDLMerge. This can take a WSDL file, local or remote, and merge it with all the XSD files referenced. The merging is recursive, so any XSD files referenced by other XSD files are also included. It can follow local path locations and remote path locations. The result is a single WSDL file, that contains everything.
If you want to jump right into the code part, you can find the tool in source code format at GoogleCode. You will need Visual Studio 2010 to compile, but you can safely run it with .NET 3.5 SP1 (maybe even earlier).
The rest of this post will talk about how this tools works.
Just XML
WSDL files are just XML files after all. So we can go ahead and load it from disk or from a URL.
And voila, we have the entire WSDL loaded up. We need to create a XML namespace manager because we are going to work with namespaces. XPath searches in particular required the namespace manager. If you have the source code by now (see link above) you can find the following code in a method named PrepareNamespaceManager().
The tools will verify if the file loaded is an actual WSDL file. It does this by checking for the root element, which should be wsdl:definitions. There are probably better ways to do this, but this is good enough for our purposes.
Schemas, where are thee?
Next step, find schemas. These can be found under the following XPath /wsdl:definitions/wsdl:types. We read the import definitions one by one, load the schema location and namespace parts of the imports. We also keep track of all namespaces we have already loaded.
Ok, so first we locate the element where we should find the schema import statements:
If such an element exists, we can start finding any schemas:
Here the process turn recursive. This is done using a method called ProcessSchema() that is designed to process a single schema definition.
Inside this method we need to know if the schema is inline or if the schema is imported. So we look for import elements:
If we find anything, we check the namespace for this schema as well as the schema location. If the namespace we find is not yet loaded, we load the .XSD file (from either disk or an URL), attach it to the main document, and remove the import statement.
The ImportNode() method handles duplicating the element from the schema into our WSDL document. We also remove any import elements from the duplicated element (this could mean removing schemaLocation attributes). We do not want any XSD to import anything.
Of course we also do not want to have any schemas missing. So while we remove schema references from the document we are creating, we will want to follow them in the original documents. After this processing done we will process the original XSD (from which we created the duplicate) for these import statements, and call ourself (ProcessSchema()) recursively to import any further XML namespaces.
When this process is complete, all namespaces (== XSD files) that are referenced in any of the directly referenced schemas or anywhere in there recursively will be included one by one in the body of the WSDL document. This sort of flattens the entire XSD structure (previously using files it was built as a tree like structure). The schema references will still be in place, and because all schemas are now in the body, the WSDL will not have any dependencies.
In the end, the whole process is just navigating and modifying XML documents, looking up references, loading them, and attaching duplicated elements and nodes into the master document. This master document will become our merged WSDL document, which we just write to disk in the end.
)
Finally, I ended up creating a tool I called WSDLMerge. This can take a WSDL file, local or remote, and merge it with all the XSD files referenced. The merging is recursive, so any XSD files referenced by other XSD files are also included. It can follow local path locations and remote path locations. The result is a single WSDL file, that contains everything.
If you want to jump right into the code part, you can find the tool in source code format at GoogleCode. You will need Visual Studio 2010 to compile, but you can safely run it with .NET 3.5 SP1 (maybe even earlier).
The rest of this post will talk about how this tools works.
Just XML
WSDL files are just XML files after all. So we can go ahead and load it from disk or from a URL.
XmlDocument wsdl = new XmlDocument (); wsdl.Load ( filename );
And voila, we have the entire WSDL loaded up. We need to create a XML namespace manager because we are going to work with namespaces. XPath searches in particular required the namespace manager. If you have the source code by now (see link above) you can find the following code in a method named PrepareNamespaceManager().
XmlNamespaceManager manager = new XmlNamespaceManager ( wsdl.NameTable );
manager.AddNamespace ( "wsdl", WSDLNamespace );
manager.AddNamespace ( "xsd", XSDNamespace );
return manager;
The tools will verify if the file loaded is an actual WSDL file. It does this by checking for the root element, which should be wsdl:definitions. There are probably better ways to do this, but this is good enough for our purposes.
Schemas, where are thee?
Next step, find schemas. These can be found under the following XPath /wsdl:definitions/wsdl:types. We read the import definitions one by one, load the schema location and namespace parts of the imports. We also keep track of all namespaces we have already loaded.
Ok, so first we locate the element where we should find the schema import statements:
XmlNode node = wsdl.SelectSingleNode ( "/wsdl:definitions/wsdl:types", manager );
If such an element exists, we can start finding any schemas:
XmlElement schemaElement = typesElement.SelectSingleNode ( "xsd:schema", manager ) as XmlElement;
Here the process turn recursive. This is done using a method called ProcessSchema() that is designed to process a single schema definition.
Inside this method we need to know if the schema is inline or if the schema is imported. So we look for import elements:
imports = rootElement.SelectNodes ( "xsd:import", manager );
If we find anything, we check the namespace for this schema as well as the schema location. If the namespace we find is not yet loaded, we load the .XSD file (from either disk or an URL), attach it to the main document, and remove the import statement.
XmlDocument schemaDocument = new XmlDocument ();
schemaDocument.Load ( importLocation );
XmlElement newSchema = wsdl.ImportNode ( schemaDocument.DocumentElement, true ) as XmlElement;
XmlNodeList newImports = newSchema.SelectNodes ( "/xsd:import", manager );
foreach ( XmlNode importNode in newImports )
{
if ( level == 0 )
{
newSchema.RemoveChild ( importNode );
}
else
{
if ( importNode.Attributes["schemaLocation"] != null )
{
importNode.Attributes.RemoveNamedItem ( "schemaLocation" );
}
}
}
schemas.Add ( importNamespace, newSchema );
The ImportNode() method handles duplicating the element from the schema into our WSDL document. We also remove any import elements from the duplicated element (this could mean removing schemaLocation attributes). We do not want any XSD to import anything.
Of course we also do not want to have any schemas missing. So while we remove schema references from the document we are creating, we will want to follow them in the original documents. After this processing done we will process the original XSD (from which we created the duplicate) for these import statements, and call ourself (ProcessSchema()) recursively to import any further XML namespaces.
When this process is complete, all namespaces (== XSD files) that are referenced in any of the directly referenced schemas or anywhere in there recursively will be included one by one in the body of the WSDL document. This sort of flattens the entire XSD structure (previously using files it was built as a tree like structure). The schema references will still be in place, and because all schemas are now in the body, the WSDL will not have any dependencies.
In the end, the whole process is just navigating and modifying XML documents, looking up references, loading them, and attaching duplicated elements and nodes into the master document. This master document will become our merged WSDL document, which we just write to disk in the end.
Plug & Play MVC
(MVC)
This entry will be available in Finnish only - it is related to the TechDays 2011 speech on 31.3 about extending MVC framework.
Pidin esityksen TechDays 2011 MVC kehikon laajennuksesta. Esityksessä käsiteltiin miten MVC kehikkoa voi laajentaa eri paikoilla.
Itse kehikko on iso kokoelma pieniä osia mitkä toimivat hyvin yhdessä. Kuitenkin näitä pieniä osia voidaan muokata ja räätälöidä, tai jopa tehdä omia osia mitkä käytetään alkuperäisten sijasta. Näin voimme räätälöidä itse kehikonkin että se toimii juuri niin kuin me haluamme.
Näitä laajennuspisteitä on monta, ja esitys yrittää antaa kartan näihin pisteisiin. Ohjeiden perusteella pitäisi olisi helppo löytää juuri se laajennuskohta, mikä antaa mahdollisimman paljon hyötyä.
Esityksen PDF kalvot ovat ladattavissa täältä.
Esityksen demo sovellus on myös ladattava täältä.
Demo vaatii Visual Studio 2010 version (Express pitäisi toimia myös) sekä SQL Server Express version. Kun sovellusta käynnistää ensimmäinen kertaa, se luo itselleen tietokannan (App_Data kansioon). Kuitenkin pitää käsin ajaa alustus. Se löytyy /Home/Setup osoitteesta sovelluksen alta (eli HomeController:in Setup action).
Pidin esityksen TechDays 2011 MVC kehikon laajennuksesta. Esityksessä käsiteltiin miten MVC kehikkoa voi laajentaa eri paikoilla.
Itse kehikko on iso kokoelma pieniä osia mitkä toimivat hyvin yhdessä. Kuitenkin näitä pieniä osia voidaan muokata ja räätälöidä, tai jopa tehdä omia osia mitkä käytetään alkuperäisten sijasta. Näin voimme räätälöidä itse kehikonkin että se toimii juuri niin kuin me haluamme.
Näitä laajennuspisteitä on monta, ja esitys yrittää antaa kartan näihin pisteisiin. Ohjeiden perusteella pitäisi olisi helppo löytää juuri se laajennuskohta, mikä antaa mahdollisimman paljon hyötyä.
Esityksen PDF kalvot ovat ladattavissa täältä.
Esityksen demo sovellus on myös ladattava täältä.
Demo vaatii Visual Studio 2010 version (Express pitäisi toimia myös) sekä SQL Server Express version. Kun sovellusta käynnistää ensimmäinen kertaa, se luo itselleen tietokannan (App_Data kansioon). Kuitenkin pitää käsin ajaa alustus. Se löytyy /Home/Setup osoitteesta sovelluksen alta (eli HomeController:in Setup action).
In response to the comments I have received to the original article, I have updated it with information how to modify the behavior of the jQuery validator plugin without directly editing the source.
While working with MVC 3, I hit some problems with validation. MVC 3 uses jQuery validation plugin with unobtrusive validation to help you out. Except when I had to use Finnish formatted numbers, there was a problem. jQuery/javascript did not really like the comma as a decimal separator.
A similar problem is with the range validation, which also refuses to work when using a comma as the decimal separator.
A rather nasty set of problems I might say, especially since number validation rules are added AUTOMATICALLY by ASP.NET MVC if you render a TextBox for a Decimal field in the model. It just seems the writer of the validator plugin forgot about everyone outside US (thanks @Deef for this last comment).
Fixing jquery.validate.js
You can of course go into the source of the validator javascript file and make your changes there, but modifying this file should never be done directly. That would create all kinds of nasty problems when updating to future versions.
Instead, one should take advantage of extending (replacing) the validator functions that are provided by jQuery Validate. (thanks to @ruben for the information on this).
To fix these problems, we can take the default implementation from the jquery.validate.js file for the range() and number() functions. We then create another .js file (say jQueryFixes.js), in which we override these default functions with ones that contain support for the comma as a decimal separator. The contents of the file should be something like this:
This should be the last .js file you include, so it overrides the behavior of the default functions from the jQuery Validator plugin.
For number validation, what we have done is replace the last dot with an expression that also accepts a comma (change is bolded):
return this.optional(element) || /^-?(?:\d+|\d{1,3}(?:,\d{3})+)(?:[\.,]\d+)?$/.test(value);
This will solve the number validation problem with the decimal point/comma. If you are dealing with thousand separator, you might need to tweak this further.
As for the range validation, I opted to simply replace the comma with a dot, and proceed normally from there on. You could of course use the jquery globalization plugin, please see the comments section for @ruben's recommendation on that one.
And that is about it. Now decimal separator commas should also be accepted by the validators.
While working with MVC 3, I hit some problems with validation. MVC 3 uses jQuery validation plugin with unobtrusive validation to help you out. Except when I had to use Finnish formatted numbers, there was a problem. jQuery/javascript did not really like the comma as a decimal separator.
A similar problem is with the range validation, which also refuses to work when using a comma as the decimal separator.
A rather nasty set of problems I might say, especially since number validation rules are added AUTOMATICALLY by ASP.NET MVC if you render a TextBox for a Decimal field in the model. It just seems the writer of the validator plugin forgot about everyone outside US (thanks @Deef for this last comment).
Fixing jquery.validate.js
You can of course go into the source of the validator javascript file and make your changes there, but modifying this file should never be done directly. That would create all kinds of nasty problems when updating to future versions.
Instead, one should take advantage of extending (replacing) the validator functions that are provided by jQuery Validate. (thanks to @ruben for the information on this).
To fix these problems, we can take the default implementation from the jquery.validate.js file for the range() and number() functions. We then create another .js file (say jQueryFixes.js), in which we override these default functions with ones that contain support for the comma as a decimal separator. The contents of the file should be something like this:
$.validator.methods.range = function (value, element, param) {
var globalizedValue = value.replace(",", ".");
return this.optional(element) || (globalizedValue >= param[0] && globalizedValue <= param[1]);
}
$.validator.methods.number = function (value, element) {
return this.optional(element) || /^-?(?:\d+|\d{1,3}(?:[\s\.,]\d{3})+)(?:[\.,]\d+)?$/.test(value);
}
This should be the last .js file you include, so it overrides the behavior of the default functions from the jQuery Validator plugin.
For number validation, what we have done is replace the last dot with an expression that also accepts a comma (change is bolded):
return this.optional(element) || /^-?(?:\d+|\d{1,3}(?:,\d{3})+)(?:[\.,]\d+)?$/.test(value);
This will solve the number validation problem with the decimal point/comma. If you are dealing with thousand separator, you might need to tweak this further.
As for the range validation, I opted to simply replace the comma with a dot, and proceed normally from there on. You could of course use the jquery globalization plugin, please see the comments section for @ruben's recommendation on that one.
And that is about it. Now decimal separator commas should also be accepted by the validators.
Some time ago I wrote an article about MVC and virtual views. I made a comment that you could create a VirtualPathProvider that would serve content from a ZIP file. Some of you commented you would like to see an implementation for that. Well, here it is.
Please read that article for more information on how to implement a VirtualPathProvider. Here I will not go into details on everything involved.
You can download the entire example solution (Visual Studio 2010 and MVC 3 required!) from here.
The example should run as soon as you load it up in VS2010. Try to request the following addresses (relative to the application root): /Test and Test/Complex for non zipped versions of views. And /TestZipped and /TestZipped/Complex for zipped versions of the views.
How to extract files from a .ZIP file?
For this example, I will use SharpZipLib, which a freely available .zip file management library. It is already included in the example solution I linked at the beginning of this article. So if you downloaded that you will not need to download SharpZipLib separately.
Preparing to serve Views from a .ZIP file
I decided that the content I want to serve will be stored in a file called Views.zip, that needs to be stored at the root of the application (~/Views.zip). This .zip file has to have the same directory structure as the real MVC application. So we need a Views folder, and inside that, the controller folder. Inside the controller folder (or folders) there will be the actual views.
For the example solution I put the views for the "TestZipped" controller into the Views.zip file. There are two views, Index.cshtml and Complex.cshtml. Both include Razor markup to demonstrate that the solution demonstrated is actually fully functional. You could of course pack up the entire Views folder in the example solution, and it will still work
(I did try this, and it did work. If you do it, make sure you still leave web.config in place, as that is outside the scope of VirtualPathProvider. But you can just put everything else in the Views/ folder into the .ZIP file, and delete them from the actual folder. The application will still work!)
The directory structure inside the .zip will make it easy to find and extract the files. Because the VirtualPathManager receives virtual file names, like /Views/TestZipped/Complex.cshtml, when we have the same structure in the .zip, it is easy to string compare the names.
Of course you could implement as complex a logic as you want to find the proper file
Creating a VirtualPathProvider for serving content from .ZIP files
The next step is to create a new VirtualPathProvider. Please note that the implementation I provide is for demonstration purposes only, and might not be fully ready for production use.
I also created a little helper method to register our path provider.
We will override FileExists(), GetFile() and GetCacheDependency(). Let's start with FileExists().
We simply want to check if we have the .zip file is available and if the file can be found in the .zip file. If both conditions are true, we can indicate we found the file. Otherwise, we let the default implementation take over and perform any checks needed.
So to check if the .zip exists, we just use the System.IO.File.Exists() method. To verify if the file exists, we read the .zip file entries and try to find one that matches our file. If we find it, we return true. Simple enough.
For getting the file, we use similar code:
Here we check if the .zip file exists, and then try to extract the file from it. If it is found, it is also returned. A new class called ZippedVirtualFile is used to return the file, this will be introduced a little later below.
Finally, we override the GetCacheDependency() method, because MVC uses that to monitor when it needs to rebuild/reread a file. What we will do is instead of returning a cache dependency on the file, we return a dependency on the .zip file. If the .zip file changes we will assume our file changes. Not very robust, but will work for this example.
ZippedVirtualFile
We need a little helper class that will hold our extracted file, and return a stream to it on demand.
The class is very simple, it simply holds a byte array, and when a stream is requested, returns a new MemoryStream that can be used to read the contents.
Using the ZippedVirtualPathProvider
As a final step, we need to register our provider. This can be done in global.asax.cs by adding the following code to the Application_Start() method:
And we are ready to go! (As soon as we create the Views.zip file of course).
Please read that article for more information on how to implement a VirtualPathProvider. Here I will not go into details on everything involved.
You can download the entire example solution (Visual Studio 2010 and MVC 3 required!) from here.
The example should run as soon as you load it up in VS2010. Try to request the following addresses (relative to the application root): /Test and Test/Complex for non zipped versions of views. And /TestZipped and /TestZipped/Complex for zipped versions of the views.
How to extract files from a .ZIP file?
For this example, I will use SharpZipLib, which a freely available .zip file management library. It is already included in the example solution I linked at the beginning of this article. So if you downloaded that you will not need to download SharpZipLib separately.
Preparing to serve Views from a .ZIP file
I decided that the content I want to serve will be stored in a file called Views.zip, that needs to be stored at the root of the application (~/Views.zip). This .zip file has to have the same directory structure as the real MVC application. So we need a Views folder, and inside that, the controller folder. Inside the controller folder (or folders) there will be the actual views.
For the example solution I put the views for the "TestZipped" controller into the Views.zip file. There are two views, Index.cshtml and Complex.cshtml. Both include Razor markup to demonstrate that the solution demonstrated is actually fully functional. You could of course pack up the entire Views folder in the example solution, and it will still work
(I did try this, and it did work. If you do it, make sure you still leave web.config in place, as that is outside the scope of VirtualPathProvider. But you can just put everything else in the Views/ folder into the .ZIP file, and delete them from the actual folder. The application will still work!
)
The directory structure inside the .zip will make it easy to find and extract the files. Because the VirtualPathManager receives virtual file names, like /Views/TestZipped/Complex.cshtml, when we have the same structure in the .zip, it is easy to string compare the names.
Of course you could implement as complex a logic as you want to find the proper file
Creating a VirtualPathProvider for serving content from .ZIP files
The next step is to create a new VirtualPathProvider. Please note that the implementation I provide is for demonstration purposes only, and might not be fully ready for production use.
public class ZippedVirtualPathProvider : VirtualPathProvider
{
public static void RegisterMe()
{
HostingEnvironment.RegisterVirtualPathProvider(new ZippedVirtualPathProvider());
}
I also created a little helper method to register our path provider.
We will override FileExists(), GetFile() and GetCacheDependency(). Let's start with FileExists().
We simply want to check if we have the .zip file is available and if the file can be found in the .zip file. If both conditions are true, we can indicate we found the file. Otherwise, we let the default implementation take over and perform any checks needed.
public override bool FileExists(string virtualPath)
{
if (IsViewsZipFound && FileExistsInZip(virtualPath))
{
return true;
}
else
{
return base.FileExists(virtualPath);
}
}
private bool IsViewsZipFound
{
get
{
if (File.Exists(ViewsZipPath))
{
return true;
}
else
{
return false;
}
}
}
private static string ViewsZipPath
{
get
{
return HttpContext.Current.Server.MapPath ( "~/Views.zip" );
}
}
private static string TransformVirtualPath2ZipPath(string virtualPath)
{
string fullPath = virtualPath;
if (fullPath.StartsWith("~/"))
{
fullPath = fullPath.Substring(2);
}
if (fullPath.StartsWith("/"))
{
fullPath = fullPath.Substring(1);
}
return fullPath;
}
private bool FileExistsInZip(string virtualPath)
{
string zipPath = TransformVirtualPath2ZipPath(virtualPath);
using (ZipInputStream s = new ZipInputStream(File.OpenRead(ViewsZipPath)))
{
ZipEntry theEntry;
while ((theEntry = s.GetNextEntry()) != null)
{
if (theEntry.IsFile && theEntry.Name == zipPath)
{
return true;
}
}
}
return false;
}
So to check if the .zip exists, we just use the System.IO.File.Exists() method. To verify if the file exists, we read the .zip file entries and try to find one that matches our file. If we find it, we return true. Simple enough.
For getting the file, we use similar code:
public override VirtualFile GetFile(string virtualPath)
{
if (IsViewsZipFound)
{
var file = ExtractFromZip(virtualPath);
if (file != null)
{
return file;
}
}
return base.GetFile(virtualPath);
}
private VirtualFile ExtractFromZip (string virtualPath)
{
string zipPath = TransformVirtualPath2ZipPath(virtualPath);
using (ZipInputStream s = new ZipInputStream(File.OpenRead(ViewsZipPath)))
{
ZipEntry theEntry;
while ((theEntry = s.GetNextEntry()) != null)
{
if (theEntry.IsFile && theEntry.Name == zipPath)
{
byte[] data = new byte[theEntry.Size];
s.Read(data, 0, data.Length);
return new ZippedVirtualFile(virtualPath, data);
}
}
}
return null;
}
Here we check if the .zip file exists, and then try to extract the file from it. If it is found, it is also returned. A new class called ZippedVirtualFile is used to return the file, this will be introduced a little later below.
Finally, we override the GetCacheDependency() method, because MVC uses that to monitor when it needs to rebuild/reread a file. What we will do is instead of returning a cache dependency on the file, we return a dependency on the .zip file. If the .zip file changes we will assume our file changes. Not very robust, but will work for this example.
public override System.Web.Caching.CacheDependency GetCacheDependency(string virtualPath, System.Collections.IEnumerable virtualPathDependencies, DateTime utcStart)
{
if (IsViewsZipFound && FileExistsInZip(virtualPath))
{
return new System.Web.Caching.CacheDependency(ViewsZipPath, utcStart);
}
else
{
return base.GetCacheDependency(virtualPath, virtualPathDependencies, utcStart);
}
}
ZippedVirtualFile
We need a little helper class that will hold our extracted file, and return a stream to it on demand.
public class ZippedVirtualFile : VirtualFile
{
public byte[] data;
public ZippedVirtualFile(string virtualPath, byte[] data)
: base(virtualPath)
{
this.data = data;
}
public override System.IO.Stream Open()
{
return new MemoryStream(data);
}
}
The class is very simple, it simply holds a byte array, and when a stream is requested, returns a new MemoryStream that can be used to read the contents.
Using the ZippedVirtualPathProvider
As a final step, we need to register our provider. This can be done in global.asax.cs by adding the following code to the Application_Start() method:
protected void Application_Start()
{
// ...
ZippedVirtualPathProvider.RegisterMe();
}
And we are ready to go! (As soon as we create the Views.zip file of course).
I was trying to find a solution to a problem with LINQ and its CreateDatabase() call (see details in another blog post). Then I bumped into the following problem.
When I tried attaching a database into an SQL Server Express User Instance that existed before (the same database under the same path), I received an exception. I actually manually deleted the .MDF and .LDF files from disc, and then tried to recreate them programatically. The exception informed me the database file already existed. Although I have deleted it myself.
The exception said: Database 'path_to_database' already exists. Choose a different database name.. Which was very strange.
Reading more about SQL Server Express User Instances, I found out that they are more complicated than I thought. What actually happens is that SQL Server Express copies the master and msdb databases under the user's directory (who runs the user instance). It then starts the user instance (sqlservr.exe) under the user account. When you want to use a database in a user instance, it actually attaches the database file to the user instance - just as you would attach a database to a regular SQL Server instance. It just happens behind the scenes.
This creates trouble of course, because the database is registered somewhere, and you shouldn't just delete the files. Which I did. And that is why I got the error message.
There is some information about a tool that you can use to connect to a user instance and execute commands (detach maybe?). I found it too complicated. (But if you are interested, visit this MSDN page for more information!)
Rather, I killed my user instance process (Task Manager, kill sqlservr.exe). Then proceeded to delete my user instance directory. For Windows 7, this would be:
c:\Users\username\AppData\Local\Microsoft\Microsoft SQL Server Data\SQLEXPRESS\
I then restarted my application (which called CreateDatabase()), and it worked. It recreated the above directory, and everything was fine.
I have to admit that this seems like a bit of a drastic solution, but it sure is faster than connecting to the instance and issuing SQL commands by hand
And it should not have any side effects, because all user instance databases will just get attached automatically the next time you use them.
When I tried attaching a database into an SQL Server Express User Instance that existed before (the same database under the same path), I received an exception. I actually manually deleted the .MDF and .LDF files from disc, and then tried to recreate them programatically. The exception informed me the database file already existed. Although I have deleted it myself.
The exception said: Database 'path_to_database' already exists. Choose a different database name.. Which was very strange.
Reading more about SQL Server Express User Instances, I found out that they are more complicated than I thought. What actually happens is that SQL Server Express copies the master and msdb databases under the user's directory (who runs the user instance). It then starts the user instance (sqlservr.exe) under the user account. When you want to use a database in a user instance, it actually attaches the database file to the user instance - just as you would attach a database to a regular SQL Server instance. It just happens behind the scenes.
This creates trouble of course, because the database is registered somewhere, and you shouldn't just delete the files. Which I did. And that is why I got the error message.
There is some information about a tool that you can use to connect to a user instance and execute commands (detach maybe?). I found it too complicated. (But if you are interested, visit this MSDN page for more information!)
Rather, I killed my user instance process (Task Manager, kill sqlservr.exe). Then proceeded to delete my user instance directory. For Windows 7, this would be:
c:\Users\username\AppData\Local\Microsoft\Microsoft SQL Server Data\SQLEXPRESS\
I then restarted my application (which called CreateDatabase()), and it worked. It recreated the above directory, and everything was fine.
I have to admit that this seems like a bit of a drastic solution, but it sure is faster than connecting to the instance and issuing SQL commands by hand
And it should not have any side effects, because all user instance databases will just get attached automatically the next time you use them.
You might be familiar with SQL Server Express User Instances. These are per user copies of the database server, into which you can attach files "on the fly", without a real need for a fully managed SQL environment. You probably bumped into it if you have used the default setup for MVC or ASP.NET where you tried to use membership or other services. ASP.NET created a .MDF and .LDF file for you in the App_Data directory automatically. These are actually SQL Server Express User Instance databases.
If you ever checked the ConnectionString, it will contain a |DataDirectory| directive (and the term User intance=true. At runtime, the ASP.NET runtime resolves this to point into the App_Data folder of the application. The SQL client runtime of .NET is able to connect up this file into the user instance and use it, without any user action needed. It can also create it, if it does not exist.
While ASP.NET provides this behavior for its own databases by default, there is nothing stopping us from using the same feature with our custom application databases.
Create a missing database
When I put code into source code control, I rarely put it actual .MDF and .LDF files (databases). Instead, usually an .SQL script goes in there that can create my database. However, since LINQ has the neat feature to create the missing database, why not use that? This way I could make my project create the database it needs by itself, when starting up.
It would be ideal for distributing demos and examples. The person who opens it up, just needs to start it. Visual Studio self hosts the web application in its own web server, the database is created in the SQL Server Express User Instance, and it just starts working.
It is all very simple in theory, until you want to call LINQ's CreateDatabase() on a LINQ model that has a ConnectionString with the |DataDirectory| variable. I used CreateDatabase() succesfully before with regular SQL server databases. But when I tried the user instance approached, it failed.
My first try was the following code:
And what I got was an ArgumentException, that told me: Illegal characters in path.. Strange. I checked the callstack, which was:
I checked everything, but all seemed to be fine.
I then tried substituting the actual path in the connection string, and my code worked. So what is going on?
Bug in LINQ DataContext
LINQ DataContext uses its own SQL wrapper, and that wrapper has a bug. It cannot resolve the |DataDirectory| constant. I used to .NET Reflector to verify this. Sure enough, the System.Data.Linq.SqlClient.SqlProvider class retrieves the AttachDbFilename entry from the connection string, and then uses it directly. So if you have |DataDirectory| in there, you are out of luck: an exception will inform you of this
Luckily, you can retrieve the value of the DataDirectory entry from the current AppDomain. So you could do the substitution yourself, pass the corrected connection string to the DataContext and create the database. Voila!
DataDirectory
So first, we need the DataDirectory entry. We can use the following code to retrieve it from the current AppDomain:
Putting it together
So I ended up writing the following method into my Global.asax.cs file:
This code loads my connection string and if it finds the |DataDirectory| directive in there, it will replace it with the DataDirectory value from the AppDomain. It then fires up an instance of the DataContext and makes sure the database exists. Sure enough, this works (except ... see below).
This code can be called in Global.asax.cs in the Application_Start() method. After this, whenever you run the application and it finds the database is missing, it will create it. You would still need to populate it with initial data, if that is a requirement, because the database will be empty of course.
Multiple user instance databases
While testing the above problem, I replace the DataDirectory variable by hand first, tried it that way, and it worked. I then stopped the web server, deleted the created files and started playing around. I bumped into a problem when I then wanted to create the files again. As if SQL server believed the files existed, although I deleted them. As it turns out, User Instances are not as simple as they sound. There is a slight problem when you want to attach a database with the same path into the user instance that already existed. I wrote a separate blog post about that. See it here.
If you ever checked the ConnectionString, it will contain a |DataDirectory| directive (and the term User intance=true. At runtime, the ASP.NET runtime resolves this to point into the App_Data folder of the application. The SQL client runtime of .NET is able to connect up this file into the user instance and use it, without any user action needed. It can also create it, if it does not exist.
While ASP.NET provides this behavior for its own databases by default, there is nothing stopping us from using the same feature with our custom application databases.
Create a missing database
When I put code into source code control, I rarely put it actual .MDF and .LDF files (databases). Instead, usually an .SQL script goes in there that can create my database. However, since LINQ has the neat feature to create the missing database, why not use that? This way I could make my project create the database it needs by itself, when starting up.
It would be ideal for distributing demos and examples. The person who opens it up, just needs to start it. Visual Studio self hosts the web application in its own web server, the database is created in the SQL Server Express User Instance, and it just starts working.
It is all very simple in theory, until you want to call LINQ's CreateDatabase() on a LINQ model that has a ConnectionString with the |DataDirectory| variable. I used CreateDatabase() succesfully before with regular SQL server databases. But when I tried the user instance approached, it failed.
My first try was the following code:
MyDataContext ctx = new MyDataContext( connString )
if (ctx.DatabaseExists() == false)
{
ctx.CreateDatabase();
}
And what I got was an ArgumentException, that told me: Illegal characters in path.. Strange. I checked the callstack, which was:
at System.IO.Path.CheckInvalidPathChars(String path) at System.IO.Path.NormalizePath(String path, Boolean fullCheck, Int32 maxPathLength) at System.IO.Path.GetFullPathInternal(String path) at System.IO.Path.GetFullPath(String path) at System.Data.Linq.SqlClient.SqlProvider.System.Data.Linq.Provider.IProvider.CreateDatabase() at System.Data.Linq.DataContext.CreateDatabase()
I checked everything, but all seemed to be fine.
I then tried substituting the actual path in the connection string, and my code worked. So what is going on?
Bug in LINQ DataContext
LINQ DataContext uses its own SQL wrapper, and that wrapper has a bug. It cannot resolve the |DataDirectory| constant. I used to .NET Reflector to verify this. Sure enough, the System.Data.Linq.SqlClient.SqlProvider class retrieves the AttachDbFilename entry from the connection string, and then uses it directly. So if you have |DataDirectory| in there, you are out of luck: an exception will inform you of this
Luckily, you can retrieve the value of the DataDirectory entry from the current AppDomain. So you could do the substitution yourself, pass the corrected connection string to the DataContext and create the database. Voila!
DataDirectory
So first, we need the DataDirectory entry. We can use the following code to retrieve it from the current AppDomain:
string dataDirectory = AppDomain.CurrentDomain.GetData("DataDirectory").ToString();
Putting it together
So I ended up writing the following method into my Global.asax.cs file:
private void EnsureDatabase()
{
string connString = null;
connString = global::System.Configuration.ConfigurationManager.ConnectionStrings["MyConnectionString"].ConnectionString;
if (connString.Contains("|DataDirectory|"))
{
string dataDirectory = AppDomain.CurrentDomain.GetData("DataDirectory").ToString();
connString = connString.Replace("|DataDirectory|", dataDirectory);
}
using ( MyDataContext ctx = new MyDataContext( connString ) )
{
if (ctx.DatabaseExists() == false)
{
ctx.CreateDatabase();
}
}
}
This code loads my connection string and if it finds the |DataDirectory| directive in there, it will replace it with the DataDirectory value from the AppDomain. It then fires up an instance of the DataContext and makes sure the database exists. Sure enough, this works (except ... see below).
This code can be called in Global.asax.cs in the Application_Start() method. After this, whenever you run the application and it finds the database is missing, it will create it. You would still need to populate it with initial data, if that is a requirement, because the database will be empty of course.
Multiple user instance databases
While testing the above problem, I replace the DataDirectory variable by hand first, tried it that way, and it worked. I then stopped the web server, deleted the created files and started playing around. I bumped into a problem when I then wanted to create the files again. As if SQL server believed the files existed, although I deleted them. As it turns out, User Instances are not as simple as they sound. There is a slight problem when you want to attach a database with the same path into the user instance that already existed. I wrote a separate blog post about that. See it here.