Daily Dribblings of a Demented Developer

The Slippery Slide

2011-11-13T17:35:00.001+11:00

I was raised as a fundamentalist Christian. Until the age of about 25, I believed the following facts to be immutable.

There is a metaphysical all knowing and perfect being who exists outside of time and space, who for want of a better name is called God.
This God being, created the entire observable universe including a species of cognisant beings in his image who at some point in their "time based" existence started referring to themselves as homo sapiens. These beings were created with a mortal physical presence and an immortal soul.
This God being, spoke to some of these cognisant beings throughout this time based history, that were deemed worthy to accurately record all of his various thoughts and actions over the course of a few thousand years. These writings were then collected by a committee of these cognisant beings around 300 ACE who were doing the will of this God being. This collection of works went by the name "The Holy Bible".
This God being made up some elaborate rules for these cognisant beings to gain this Gods favour and be eligible to co-exist with this God being after there mortal physical presence had expired, in a really nice place. The alternative to this really nice place was a pretty horrible place, and those who didn't follow these elaborate rules would be sent there. These elaborate rules were spelled out in this Holy Bible book. Oh and
This God being has a penis... (A freakin' huge one).

I chose the word "fundamentalist" in my opening sentence very specifically. There is a quintessential difference between someone who believes

The Bible is the written word of God and infallible, and someone who believes that
The Bible is a book of collected works written by humans who were fallible, but had certain insights into the divine at a particular point in history.

There are many aspects in which this distinction plays itself out. One we see commonly, is the biblical literalists insistence on attacking the scientific theory of evolution, and insisting, against all evidence to the contrary, that the earth is less than 10,000 years old. The reason Biblical literalists cannot, and will not accept that the creation story is anything more than a myth for the benefit of an illiterate desert tribe to take solace in, is because they argue that this denial in the literal truth of the opening chapter of the Bible creates a slippery slide that will hold the rest of the Bibles message to critique, and will ultimately invalidate (in their minds) the undeniable truth of the word of God. It has very little to do with the psuedo-science of intelligent design. People who fall into category b however, (those I would refer to as liberal Christians), see no problem with juxtaposing the creation myth as a beautiful and metaphorically elegant story of the supernatural origins of the universe against the scientific theory of evolution. I myself went through a phase of believing that the Bible wasn't meant to be a modern scientific dissertation, but rather in regards to the creation story was meant to provide sufficient information for people in a pre-scientific era to satisfy the ontological question, and move beyond this to more important aspects of how to live a good life. While I was still able to accept the rest of the dogmas of the particular brand of Christianity I was involved with at the time, this event marked my slow progression from fundamentalist Christian to liberal Christian, which eventually culminated in my current position of Atheism. But there was much more to this progression than simply taking the creation story metaphorically.

I've spoken with a number of ex-fundamentalists. Some have, like me, gone to almost the polar opposite of atheism. Others have chosen the less extreme agnostic route. Some have found solace in some form of deism, while others prefer to hold to their theistic understanding of the world but chose a more liberal approach to their Christianity. Many of these people, however, will have some key thought, argument or idea that has eventually broken the continuum of the cyclic logic that is necessary to maintain a fundamentalist belief system. What I wish to do in this essay is to explain what that argument was for me, and how I was finally able to break out of the fundamentalist mind-set. Note to any fundamentalists who might be reading this essay, before you start formulating your killer cyclic argument that will in your mind destroy my "sole" objection to your fundamentalist belief, just be warned that this is not my "sole" objection, but merely the one that I deem responsible for enabling me to jettison my dogmatic and limited world view.

Language

Before I go on, I need to say something about language, and my understanding of it. I have always had an interest in languages. If you haven't figured out by now, English is my native language, and despite attempts at learning a few others, I have to concede that I am not yet bi-lingual, although it remains one of my life goals to become so. In high school I did a few years of Italian, not that I remember much of it now. I have had a number of attempts at learning Spanish, and it is currently the language I am most fluent in aside of English. I can actually hold decent conversations in Spanish, and understand a reasonable amount. I even attempted to learn Greek at one stage. Very early on in my investigation of the concepts of language, and the ways in which languages differ, I started to feel very overwhelmed with the way in which the language we speak, and the way in which we use language can shape our subjective experience and understanding of the world. This can be demonstrated using a very simple example such as the following.

In English we would say

"The red shirt"

The equivalent Spanish translation would be

"La camisa roja"

Even in this very simple 3 word example we notice two very astounding things. Firstly the word ordering is different. In English we have the definite article "the" followed by the adjective "red" and finally the noun "shirt". In Spanish we have the definite article "la" followed immediately by the noun "camisa" and then finally the adjective "roja". The second and even more significant difference is that in Spanish nouns have a gender whereas in English they are gender neutral. The word for shirt in Spanish "camisa" ends in an "a" making this word feminine, and forcing the usage feminine definite article "la" instead of "el"(masculine) as well as the feminine form of the adjective for red "roja" instead of "rojo".

These differences may seem small, and perhaps just an annoying little peculiarity if you are trying to learn a enough of Spanish so that you can order a meal at a restaurant when you go on holidays. However, if you look a little deeper, it raises all sorts of questions around how the world is viewed by the native speakers of these different languages, and how the language you use to think in shapes your subjective reality. The ordering of words is important to the emphasis we place on things, and concepts. Attributing a gender to nouns, or not changes your whole understanding of that concept, and forms different associations in the neural pathways of the brain. Admittedly, in the very simple example I give here, we a reasonably certain that we are talking about the same thing, but this is an extremely simplistic example. It doesn't take too long into trying to explain more detailed concepts that you very quickly see a difference in the ways in which people with different languages understand and perceive the world. This fact is demonstrated quite profoundly by recent scientific research that shows how our concept of languages affects the colours we perceive and how well we perceive them.

Spanish and English aren't even all that far apart in their linguistic origins. Extrapolate these differences between say Hindi or Mandarin and English, and the mind can only begin to imagine how differently these people must perceive the world and relate to it. Added to this, our language is constantly changing (I would normally use the word "evolving", but I didn't want to alienate any creationists that might be reading). Just reading a play by William Shakespeare will demonstrate just how much the English language has changed in the past 400 years. It has changed so much that learned Academics debate furiously about the meaning of certain words and phrases that subtly (or sometimes not so subtly) alter the understood meaning of the plays.

The Holy Book

This brings me to the main intent of this essay. The argument that caused me to break the cyclic logic of fundamentalism, and allowed me the freedom to intellectually assess the evidence for myself and more honestly strive for an understanding of the mysteries of life. Or as a fundamentalist Christian might put, the lie that the devil sold me to win my soul and take me to a hellish eternity.

The bible is a collection of works with many different human authors over a period of a few thousand years. Or again if you’re a fundamentalist Christian, the Bible is a collection of works all authored by God but entrusted to a number of different humans over a period of a few thousand years. Regardless of your position on this, the defining argument for fundamentalists is that "The Bible is the written word of a perfect God and therefore infallible". OK, so let's break this down. I don't even want to get into the debate here about the inconsistencies and contradictions in the Bible which are well documented. I will instead assume that if you are a biblical literalist, then you will have arguments against these supposed "contradictions", or you are blissfully ignorant of them and don't care, in which case I'm surprised you've read this far.

So from the definitions here, Firstly God is "perfect". This by definition, at least in my mind, means that God when communicating with say the angels or even just thinking aloud to himself, or creating the universe by his "word", would not use an imperfect human language to perform this act of divine communication. So God's native concept of "language" would be drastically different to anything any human actually speaks. OK then, but God wanted to communicate with us his created cognisant beings to explain his plan for our lives, and his elaborate rules for gaining salvation and entering into eternity in the nice place as opposed to the horrible place once our mortal bodies had stopped functioning. This would seem an important thing for such a God to want to do. He decides that the best way to do this is to reveal it to various people over a few thousand years in their native language. Right here is where I have my first problem. Translation is imperfect. It is notoriously easy to translate things incorrectly even between very similar human languages let alone between a perfect divine language and an imperfect human language. There is an old saying "The translator lies". So even in the original Ancient Hebrew, Ancient Aramaic and Ancient Greek that the collection of books we know as the bible was written in, there is one level of translation involved. To get to where we are today though, which is to say to a Bible written in our modern languages, involves at the least one more step of translation from one of these ancient languages which are no longer in normal every day use, and are now more for academic interest, into our modern language. The sheer amount of opportunity for inaccuracy in this step renders the task frightfully problematic. This is evidenced by the sheer number of different versions of the bible that are available in English alone, and the constant debates between different adherents of their favourite translations. At this point my belief that what we call the Bible has any claim to being "the written word of God" crumbled and shattered into pieces.

One might be tempted to argue that even though their are different translations the underlying doctrinal beliefs that come from the Bible still hold true. In response, I would simply point to the number of different denominations and factions in the Christian church today. If God really thought it was that vital to have a core set of doctrines then he's doing an extremely rotten job of communicating it to modern humans.

Life after Biblical Literalism

This was the start of the slippery slide that made me question everything I ever believed about Christianity. It made me realise that the Bible should not be given special treatment and be elevated above the critique given to any other historical document. It gave me the freedom to stop taking everything in the Bible as literal truth and to be able to understand it as a collection of books written in a particular period of time for very specific purposes that the people in those times had a far better chance of understanding than I ever will, with my language and my subjective understanding of reality. That's not to say that I don’t still find parts of the bible unbelievably inspiring and enlightening. The elegance of Jesus' words "let he who is without sin cast the first stone" is amazingly poignant and very applicable to our lives today. The problem is, the Bible is statically locked in time and place, and as a consequence there are a great many things for which it does not and cannot speak. Biblical literalists try desperately to extrapolate, but in my opinion just end up causing pain both for there own adherents and for others who they try to inflict their out-dated ideas onto.

One of the things that bothers me about my whole journey to this point, is why it took me so long. It wasn't until I was 25 that I really started to ask first-order questions about the faith I was raised with. The only conclusion I can come to is that the combination of the brainwashing working so well, and my own personal insecurities as a young adult caused me to want to keep believing it for so long.

.Net Performance and Scalability Tuning Tip #2

2011-11-12T12:09:00.001+11:00

The Database Bottleneck

Why is my database so flaming slow?

Have you heard this before? I have. I’ve also heard it as a good reason not to use relational databases. There are a number of reasons why a particular database could be slow, but lets take a step back and try to understand what a relational database is actually trying to do. Relational databases are designed to be a mathematically structured, transactionally consistent and optimized way to store and retrieve large amounts of “relational” data. If you are not storing relational data, then don’t use a relational database, it is not optimized for your usage scenario. Problem solved… well, perhaps not.

Let’s assume you are storing relational data, but the database still seems slow. Well, I could go in to detail about database tuning, but I’m not a DBA, and there are people out there who are far better at that than I. Also database tuning tends to get very vendor specific at its deepest levels. Besides, that’s not what I really want to focus on here. Maybe you could look at the performance of your Network, or the disks and other hardware your database server is running on, but again, I’m not network or hardware engineer, I’m a software engineer. So let’s fix it in software… or at least, to the best of our ability. I’m going to assume then that your database is well tuned and maintained by someone who is experienced in doing so, and knows all the different ways in your organization that this database is going to be used. Once we make these assumptions, then we can no longer say “the database is slow” I would go further and say that any modern database is actually very efficient for what it does. Still I have seen developers in this case trying to blame the database for the poor performance of the application.

Most often the database resides on another box entirely to the one you’re application is running on. This is where the problem begins. To actually get data from a database you have to create a database connection, that then opens a network port, which then negotiates a conversation over a protocol, asks the database server for some data. The database engine then does some processing to find your requested data or perform your operation, and finally returns the results as data packets over the network back to your application. Now that mightn’t seem like much, but compared to grabbing that same data from your local memory, or even a file on your local disk, we are talking many orders of magnitude difference. Even if you are using a database engine on your local machine, it will almost certainly be running in a separate process which means that although you don’t have a physical network to navigate, you still have inter-process communications protocols to deal with, and this can still be a few orders of magnitude difference to getting the data directly from memory.

The most important thing about developing against a database is to have respect for the overhead required when querying the database. Based on this respect we can establish a few basic rules about how to interact with a database.

1. Don’t keep asking the database for the same thing over and over again. If it’s something you need regularly cache it. Caching comes with its own problems, the biggest of them being when to invalidate it, but the one thing you can be certain of is that it won’t be slow.

2. Try to ask the database as few times as possible for information during any one operation. Ideally it would be great if you could simply have a single SQL statement that gets everything you need to perform the desired operation, and it is worth investing a few (thousand) CPU cycles trying to figure out exactly what data you are going to need for a particular operation before hitting the database. It is also usually better to return a little more data than you might need, than to find you have to hit the database again because you forgot something. Of course this can be taken too far, so be sensible here, you shouldn’t be returning your entire customer and products tables just to process a single order.

3. Learn SQL, or your vendor specific implementation of it at least. Understand how to form the queries that get you the data you want, and take advantage of the well tuned database (assuming it’s well tuned). If you are using a tool that generates SQL for you, like LINQ2SQL or Entity Framework, understand how it generates SQL. These tools tend to produce sub-optimal SQL because they are trying to solve very generic problems, however, they can be coaxed into doing things more efficiently if you understand how they work.

If you still think the database is slow, I would suggest doing some profiling of your application. SQL Server comes with a profiler that is more than adequate. Other database engines have their own profiling tools. The Redgate ANTS performance profiler I talked about in my previous post has a SQL profiler (warning it doesn’t work with SQL Express), that is integrated with the code performance analyser. Also just a hint if you’re using ANTS performance profiler, you need to look at the “Wall Clock” setting when you are analysing a piece of code that has a lot of SQL interaction as the CPU clock will give you a false impression.

There are many other possible performance issues that can occur with databases, but this is all I really want to talk about at this stage. I have seen many projects in my time that have suffered because the 3 rules above have not been followed.

Travelling Again

2011-09-12T09:56:00.001+10:00

I am currently travelling overseas, partly for work partly for pleasure… actually mostly for pleasure. As with other times, I find that travelling to foreign countries creates a lot to reflect on, and consequently write about. My first stop will be California. I am currently in a hotel in Los Angeles, on Monday I will be driving to Anaheim where I will be attending the Microsoft Build Windows conference, then I am planning to head to Santa Barbra for a bit of R & R. After that I will be visiting Buenos Aires. I will use this my personal blog as essentially a travel diary, but my geek blog as a means of recording my thoughts on the conference.

Microsoft doesn’t care about feedback from Australians

2011-09-11T10:33:00.001+10:00

I’m attending the Build Windows 2011 Conference in Anaheim this coming week, and as part of this Microsoft sent me out an email (presumably to all attendees, or at least all who indicated that they were a developer) inviting them to share feedback in special limited numbers discussion groups. Firstly though, they require you to fill out a quick survey. So I click on the provided link, and I’m presented with the following question.

OK, fine, I’m not too proud to be lumped in with “the rest of the world” team, so I click “Somewhere Else”, then “Next”. I am then presented with this

Thank you for your interest in our survey. Those are all the questions we have today. We hope that you will consider participating in future surveys.

OK, so basically, if you are not from any of these 8 countries, then your opinion is completely irrelevant to Microsoft. Regardless of the fact that you are the technical architect for Windows/PC version of a product that has over 400,000 user licenses world wide. Regardless of the fact that you are fostering a product that is responsible for direct license sales of SQL Server 2008 R2. The message is loud and clear, if you’re not in Microsoft’s G8, then your opinion is worthless. I wonder if our Mac Technical Architect (when he is eventually appointed) will get the same kind of treatment from Apple?

Really, the least they could have done is to let me continue with the survey, and then disregard the information anyway, I mean it really doesn’t hurt to store a few extra bytes.

Today’s Random Grumble

2011-09-02T11:51:00.001+10:00

I absolutely love OneNote, and use it all the time, but there are some things that really infuriate me at times… I mean how difficult is it really to get url parsing and highlighting working properly?

.Net Performance and Scalability Tuning Tip #1

2011-08-20T16:45:00.001+10:00

Measure twice, cut as many times as you like.

As promised, I am starting a series of blog posts on .Net performance and scalability tuning. I have been doing this in my current role at QSR International, and have been amazed at just how many possibilities I have found to improve the performance of the application. Each time I get to a point where I think I can’t get any more performance out of it, I find something else, or have another idea. Not all of these tips are specifically .Net related, many of them will be applicable to any programming language, but my focus for the past 7 years of my career has been specifically on .Net, so naturally, I will be focussing fairly heavily on that. These tips will also be fairly heavily “Rich Client” based as that's where I have spent most of my career, however some of these tips will also apply to any .Net code running anywhere.

My first tip is simple. Measure what you are trying to tune. You’ll probably see many bugs in your bug tracking system (you do use a bug tracking system right? If not I suggest you stop reading this article right now, and go find yourself a bug tracking system ASAP), that read “Application is slow when I do XYZ” or “Opening form foo takes forever”. The first step is to get this quantified. Exactly how many seconds does it take to do ‘X’, where ‘X’ is repeatable. Testers will probably have a set of test data, and you need to get your hands on this. Often performance issues surface because the data is structured in a specific way, and if you have differently structured data you may well not spot the problem (have a free “Works on My Machine Certification”). Once your testers have measured it, then you might want to start a discussion around what is acceptable, what you should be aiming for in the tuning etc….

Now I said “Measure twice”, well, after the testers have got some basic timings that you can use for comparisons at the completion of your tuning, then it’s your turn to measure. There are a number of tools that will analyse your code and assist in highlighting the areas that require the most attention. Visual Studio comes with a performance wizard under the “Analyze” menu. This is a reasonably good starting point, but personally I prefer tools with a nicer UI. My current favourite at the moment is “ANTS Performance Profiler” from redgate, but there are others out there (Feel free to leave a comment if you have another preference).

For me the visualizations provided by ANTS Performance Profiler allow me to very quickly and effectively focus my attention. Without this knowledge, you can spend a lot of time optimizing bits of code that are called so infrequently that even optimized won’t make any noticeable difference to the overall performance. Once you’ve picked a measuring tool, learn it and master it. I usually like to take a series of measurements as I go. As you improve the performance of one problematic area, others will start to rear their ugly heads. Also a series of good visualizations can make for some good discussions with management if they want to know how you’re going. it is also a good education tool for other developers in your team. You can show them exactly why you should, or shouldn’t do particular things.

Now that you have measured twice, unlike carpentry where the proverb “Measure twice, cut once” comes from, as coders, we can cut as many times as we need. Assuming you are using a decent source control system (and by the way, if you’re not, stop reading this article and go get yourself one NOW), you can confidently try different options to your hearts content. Keep in mind that sometimes release code behaves slightly differently to debug code, and as such, your final sign off should come from the testers testing on the release version of your build.

Stay tuned, much more to come.

Long Time No Blog

2011-08-20T15:33:00.001+10:00

For various reasons, I haven’t blogged much over the past few years, but now I feel inspired to get back into it. A lot has changed in my professional life since I last put pen to paper (or rather ascii to screen), but a lot still stays the same.

My last post “Enjoying my iPhone” is an indication of just how long it has been. I now use a Windows Phone 7 device, and I am really liking it. To me the iPhone UX was starting to feel a little stale, and while it was revolutionary for its time, I think MS have made a real challenge with their “metro themed” UX. Having said that, I still keep my old iPhone around just in case. My biggest gripe with my HTC HD7 is that it only has 16GB of memory (my iPhone 3GS had 32GB), and as such I can’t store my entire music collection on it.

The biggest news however, is that I no longer work for Readify. As much as I love Readify, and felt honoured working along side some of Australia’s and even the world’s best geeks, I got to a point where I really needed something different. I now work for an ISV called QSR International. They write an awesome piece of software for qualitative data research called NVivo that is used in universities and research institutes all around the world. I have been working with QSR since April this year, and so far really enjoying it. It’s partly the work that I have been doing with them that has inspired me to post a series of new blog articles on .Net performance tuning, so keep a look out for that.

In other non-geek related news, the inadequacies of the Melbourne public transport system have finally caused me to give up on my hippy car-free lifestyle, and join the masses who are forced to own a car… well almost.

I am now the proud owner of a Honda CBR 600 F4i (named Xena), and absolutely love her. Although I do still catch the bus a fair bit as I believe that it increases significantly my chances of surviving the journey.

Keep a eye out for my series on .Net performance tuning… coming soon… I promise.

Enjoying my iPhone

2010-09-07T21:21:00.001+10:00

I have known for a long while now that there were significant deficiencies in the Windows Mobile approach to smart devices. This had been made clear from Apples paradigm shifting release of the iPhone. However, the fact that I could develop applications for my windows mobile device using tools I was familiar with, kept me holding firmly to the inferior device, until about a month ago, when frustrations with my HTC touch HD (WM 6.1) reached a critical mass, and it happened to coincide with a colleague of mine upgrading to iPhone 4 and selling his old (almost unused) iPhone 3GS at a very reasonable price.
Since then, I have to say I have really been enjoying my iPhone, although, fear not dear reader, I am not about to become yet another Mac fanboi... At least not that easily.
There are some really nice things about the iPhone, but at the same time, some things that could be improved.
The Good
The focus on user experience is really where the iPhone (and indeed apple) really shine. It is quite obvious that this is a device designed to be used while traveling on a train with one hand full of groceries, and you texting arm wrapped around a grab rail for support. The different context sensitive keyboards, the size of the keyboards, the auto-word prompt when you mistype a word, the ability for applications to switch orientation all add up to a great user experience when inputting and reading text, as is evident from the fact that I can type up this whole blog post using my iPhone. The concentration on more natural user interfaces is very welcome.
The hardware is fast becoming the minimum spec for smart phone manufacturers that want to compete on the modern market, which I think signifies that apple really nailed it.
The proliferation of iPhone apps has seen the platform become an instant hit, and overall, I am extremely happy with it, to the point where I will not be going back to a windows mobile 6.* device. The overall consistency of "most" applications in the app store, and the new device friendly controls that apple have encouraged developers to use are on the whole really good, however, that could become a double edged sword.
The bad
Battery life sucks. Also coming from a palm is background, I do miss graffiti as a form of input, but I guess there were so few people that bothered to learn it that I'm probably on the minority here. The 20MB download limit when not on a wifi network I see as an affront to my free market rights to purchase the bandwidth I want and how I want it delivered.

I will be getting a Windows Phone 7 device when they come out, because I really want to get into the WP7 development space. I'll have to wait and see how well then WP7 performs
Before I make a choice as to which device I want to use on a day to day basis... May the best phone win.

You know your system is in trouble when…

2010-03-07T11:52:00.001+11:00

Tried to use my favourite short cut this morning (Windows Key + E), and I got this.

Speaking at Victoria.Net SIG

2010-03-06T12:35:00.001+11:00

Just a quick note to let everyone know that I will be talking at the Victoria .Net SIG on the Microsoft Sync Framework this coming Tuesday http://www.victoriadotnet.com.au/vic-victorianet-dev-sig-formed/vic-victorianet-dev-sig-events.aspx

Hope to see you there.

More bad luck

2010-01-30T12:15:00.001+11:00

As my attempts to learn Sync Framework continue, I find even more issues along the way, this time with the SQLCE runtime. The basic scenario boils down to attempting to fill a Detail view from a dataset linked to a SQL Compact Edition datasource. When the application is run, an error message to the effect of

Unable to load DLL 'sqlceme35.dll': The specified module could not be found. (Exception from HRESULT: 0x8007007E).

is displayed, and the application crashes. After trying many different things to solve the problem, a simple google search on the exact error yields the solution. Basically it is a problem on Vista and Win7 x64 environments, and simply requires you to install SQl Compact Edition SP1 found here http://www.microsoft.com/downloads/details.aspx?FamilyId=DC614AEE-7E1C-4881-9C32-3A6CE53384D9&displaylang=en#filelist.

I guess I was just surprised that it was not included as part of Visual Studio 2008 SP1.

A colleague of mine Mitch Denny has told me that “If I was going to pick a person to clear a minefield you would be that person”, I replied that I was hoping it meant he believed I had special unique skills for finding and defusing mines, and not just that I am expendable.

Technorati Tags: SQL Compact Edition,Visual Studio 2008

TFS 2010 Console Application and the .Net 4.0 Client Profile

2010-01-27T17:11:00.001+11:00

One little frustration that has caught me twice now (and so it’s time to blog it so I remember it), is that when you create a Console Application and then start adding references to any of the TFS API libraries to it (such as Microsoft.TeamFoundation.Client.dll), if yoiu just accept all the defaults for a Console Application in Visual Studio 2010 Beta 2, you get this warning

Warning 1 The referenced assembly "Microsoft.TeamFoundation.Client, Version=10.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a, processorArchitecture=MSIL" could not be resolved because it has a dependency on "System.Web, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" which is not in the currently targeted framework ".NETFramework,Version=v4.0,Profile=Client". BasicEventListener

and the application will not compile.

The issue is the .Net 4.0 Client Profile does not contain the reference to System.Web, which is required by the TFS API. This is simple to fix, just go to the Properties of your project, and for the Target Framework, select “.Net Framework 4” instead of the “.Net Framework 4 Client Profile”.

The unluckiest developer alive

2010-01-27T10:39:00.001+11:00

Sometimes I think I should be a tester for developer tools. For some reason I have the uncanny knack of stumbling onto the most obscure bugs in Visual Studio.

I am currently trying to learn Microsoft Sync Framework, and decided (after watching a good video on how to do it) to set up an ADO.Net Sync Services Local Database Cache. I went through the process step by step, the only 2 differences between my environment and the one demonstrated in the video was that I was running SQL Server 2008, and I was using the AdventureWorks2008 database.

The procedure I followed was this…

Create a new Winforms Application
Add A new Local Database Cache (this starts the wizard)
select the AdventureWorks database
Select the Production.Products, Production.ProductDescription, Production.ProductPhoto tables to add to the cache.
Click OK.

Everything else I left as the default configuration. The result:

So I figure it’s something to do with my environment. I spent about half an hour Googling and looking at forums before I decided that I must be the first person ever to hit this bug. Next I tried to see if it was because of something in my environment, I checked my setup

Windows 7 64 bit; SQL Server 2008 SP1; Visual Studio 2008 SP1; Sync Framework 2.0 x64;

The only thing I could think might be an issue was the Sync Framework 2.0 x64, so I uninstalled and installed the x86 version. Still the same problem.

I then opened SQL Profiler and tried to see if anything was obvious there, but concluded that it was all OK (I later discovered that I didn’t look closely enough at this).

I then cracked open the wizard dlls in reflector to see if there was any help there, but alas, theat was pretty much a dead end too.

I even posted an article on the Sync Framework Forum.

Frustrated, I decided that it had to be my own environment that was causing the problem, so I set about creating a virtual environment on GoGrid. Once I had managed to get everything installed, I tried just the Production.Products table this time, and got the exact same error.

I then tried creating a completely new table in a completely new database, and, surprise surprise, it actually worked… well, almost. It got past the wizard, but the code it generated would not compile. It appears that I stumbled on yet another bug in the Sync Framework wizard, if you use a table called “SyncTable”, the generated code is broken.

Encouraged by the “success”, I pressed on undeterred. I finally got everything working by creating a table called “MyTable”.

Not satisfied that I had made it work (finally), I decided to try and track down the bug. After about an hour or so of playing around and not having any success, I finally cracked open SQL Profiler again and found that the last statement executed before it died when trying to create a local database cache for the Production.Product table was the following

SELECT * FROM [Product]ion.[Product]

Obviously the code that works out the schema and table name needs some reworking.

Technorati Tags: ADO.Net,ADO.Net Sync Services,Sync Framework

Using the TFS 2010 SDK (Beta 2)

2010-01-09T12:10:00.001+11:00

With significant changes to the way TFS 2010 works, especially in relation to Team Build and the new concept of Projects Collections, the API has necessarily adapted to account for these changes. For developers wanting to use these new API’s, there is currently very little in the way of documentation. The TFS SDK code gallery site has a small amount of examples, as well as the beginnings of the API documentation, which although sparse, because of reasonably clear naming conventions actually offers users some guidance when attempting to write integration software for TFS 2010. There are also a few other bloggers around who have some posted code examples.

I have been looking at re-writing TFS Dependency Replicator for TFS 2010, and in so doing have created a very simple application to enumerate some of the objects I am interested in.

In the interests of helping other developers, here is my source code. You can also download the full solution from here.

To run it, you must have .Net 4.0 beta 2 and TFS 2010 Explorer Beta 2 installed on your machine.

   class Program
{
    static void Main(string[] args)
    {                        

        // Get a list of all registered Project Collections
        Console.WriteLine("Project Collections");
        var collections = RegisteredInstances.GetProjectCollections();

        foreach (var collection in collections)
        {
            Console.WriteLine("Connecting to TFS Server {0} at {1}", collection.Name, collection.Uri.ToString());

            // connect to tfs
            using (var tfs = new TeamFoundationServer(collection.Uri))
            {
                tfs.EnsureAuthenticated();
                Console.WriteLine("Successfully connected to {0}", tfs.Uri.ToString());

                // Get items of interest from TFS 
                EnumerateServer(tfs);
            }
        }

        Console.ReadLine();
    }

    private static void EnumerateServer(TeamFoundationServer tfs)
    {
        // Get all of the Team Projects
        var structureService = tfs.GetService();
        var teamProjects = structureService.ListAllProjects();

        Console.WriteLine("Team Projects");

        foreach (var teamProject in teamProjects)
        {
            Console.WriteLine("\tName: {0}", teamProject.Name);
        }
        
        // extract all the work items            
        // NOTE: this uses wiql to query the work item store
        Console.WriteLine("Work Items");
        var workItemStore = tfs.GetService();            
        var workItems = workItemStore.Query(
            "SELECT [ID], [Title] FROM WorkItems");

        foreach (WorkItem workItem in workItems)
        {
            Console.WriteLine("\tID: {0}\tTitle: {1}", workItem.Id, workItem.Title);
        }

        // Get the build servers            
        Console.WriteLine("Build Servers");
        var buildServer = tfs.GetService();
        var buildControllers = buildServer.QueryBuildControllers(true);
        
        // Get all the build controllerrs
        foreach (var buildController in buildControllers)
        {
            Console.WriteLine("\tController: {0}", buildController.Name);

            // Get all the build agents
            foreach (var agent in buildController.Agents)
            {
                Console.WriteLine("\t\tAgent: {0}, Team Project: {1}", agent.Name, agent.TeamProject);
            }
        }
       
        foreach (var teamProject in teamProjects)
        {
            // Get each build definition for this Team Project
            var buildDefinitions = buildServer.QueryBuildDefinitions(teamProject.Name);

            foreach (var buildDefinition in buildDefinitions)
            {
                Console.WriteLine("\tBuild: {0}", buildDefinition.Name);

                // Get all build Details for this build definition
                var buildDetails = buildDefinition.QueryBuilds();

                foreach (var buildDetail in buildDetails)
                {
                    Console.WriteLine("\t\tBuild Number: {0} Build Status: {1}", buildDetail.BuildNumber, buildDetail.Status);
                }
            }
        }

    }
}

Broken Angels book review

2009-12-27T18:14:00.001+11:00

Broken Angels by Richard Morgan

In his first book, "Altered Carbon", Richard Morgan sets up a universe in which human consciousness has been digitized, and can be transported over inter-planetary distances, re-sleeved (placed into another human body), and backed up. Broken Angels follows the main character Takeshi Kovacs, into another "Subjective" life on another colonized planet.

The exploration of the digitized consciousness concept is great. I particularly love the graphic description of the "Soul Market". He also does reasonable job of describing the not so subtle links between greedy corporations and wars, but I feel he could have gone further. This time Aliens make an appearance (although long extinct), and there are some interesting explorations of the academic attempts to explore alien culture, and the parallels with our attempts to understand ancient earth cultures is fairly obvious.

The development of some of the characters was good enough to understand their motives, but still allowed for some surprises in the end. Other characters I feel weren't really developed as well as they should have been.

I feel that the books main problems revolve around the central character Kovacs. At times it can feel very much a "One man takes on the world", "Alpha Male" stereotype. I think this was also the first books main problem as well. It also uses the created universe of digitized consciousness and technological advances to allow for more graphic violence, but I think that is an acceptable exploration of the ideas the universe is based on.

As with the first book, the general outlook is bleak, basically accepting that technology will never be a panacea for human ills, but it does leave the reader with the slightest glimmer of hope at least for some of the characters you have met along the way.

I will definitely be reading the third in the series.

Technorati Tags: Science Fiction,Book review

Linq2SQL Bug with derived classes and Calculated fields

2009-12-17T23:53:00.001+11:00

Today I discovered a bug in Linq2SQL. When you define a class hierarchy in dbml, where you derive a class from a base Class, and both use the same underlying table, and a discriminator property, as shown below:

If you then wish to expose a calculated field in the derived class, (and trust me I have come across situations where this is a requirement), you will find that Linq2SQL throws an exception when you attempt to insert or update items of the derived class to the database.

The exception is as follows

Test method DerivedClassTest.DerivedClassTest.TestInsert threw exception: System.ArgumentException: Property 'System.String CalculatedField' is not defined for type 'DerivedClassTest.BaseCLass'.

This bug occurs deep within the Linq2SQL logic, and is to do with the AutoSync functionality where Linq2SQL attempts to create a select statement to return the values of AutoSync fields at the same time as it performs the insert.

You may be able to work around this by setting the AutoSync property on the Column Attribute to “Never” as shown below

[Column(Storage="_CalculatedField", AutoSync=AutoSync.Never, DbType="NVarChar(61)", IsDbGenerated=true, UpdateCheck=UpdateCheck.Never)]
public string CalculatedField

And note that setting this in the designer doesn’t seem to work either, it just removes the AutoSync parameter all together, but still fails. You manually have to set it in the generated code (another bug perhaps?).

Of course this means that you won’t get the new value of the field after an insert or update to the table, and you’ll have to re-query the table if you need it.

The alternative workaround is to ensure that all calculated fields are placed on the base class.

Technorati Tags: Linq2SQL,dbml,Calculated Fields,SQL

I have joined the e-Book Revolution

2009-12-12T17:05:00.001+11:00

I have finally bitten the bullet and purchased a kindle. I had been tossing up between the kindle and the nook, and even though in many respects the nook is a superior device, there were really 2 “features” that persuaded me in the end. The first was Text To Speech, the second was availability.

The Nook does not have Text To Speech, and I could not find any evidence that they were thinking of adding it in a firmware upgrade. Also I wanted it before Christmas, and the earliest anyone can get there hands on a Nook is 15th of January.

I’ve had my kindle for 2 days now, and so far I am really happy with my purchase, but there are certainly some negatives, so here is my review.

The positives.

Great looking device, thin and light.
reasonable 3G coverage (in most major Australian cities).
Text to speech rocks.
Latest firmware upgrade (2.3) adds native support pdfs.
Good reading experience (eInk is cool).
Support for converting files of many different formats such as html, mobi pocket, rtf, etc…, although native support for these document types would be better.
Can sync your personal documents via your own PC.

The negatives

No Wifi. I have been amazed at how Amazon have actually sold this as a feature “You don’t have to hunt around for a wifi hotspot”, actually Amazon, I have one of them at home, as do many people, also many of the cafe’s I would go to read have them as well.
No Australian news content. This really infuriates me, I would love to get The Age delivered to my kindle, but alas I’ll have to go on killing trees to get my news.
Proprietary DRM format. It really annoys me that if I do eventually decide to go for another ebook reading device, I will potentially lose my entire book collection.
Not all books come with Text To Speech switched on. I got caught out on this on my first purchase, not checking the information thoroughly enough. I feel a bit cheated, as I feel the publisher has no right to determine how I choose to consume their content. Are they going to dictate that I cannot have another human being read it to me? No, then why draw the line at a computer?

My particular reason for wanting the Text To Speech feature is that I am a very slow reader, and Text To Speech would drastically increase the amount of books I could “read”. In future I will be making purchasing decisions based heavily on whether or not Text To Speech has been allowed by the publisher. Having said that, I wouldn’t be surprised if the problem with Text To Speech rights lies more with audio book retailers than with publishers, but that’s one for the conspiracy theorists to argue about.

Visual Studio Help Integration Wizard and Continuous Integration

2009-12-03T18:28:00.001+11:00

As .Net developers we are constantly using msdn and the Visual Studio help system. One of the most powerful features of Visual Studio is the ability to hit the F1 key on a class, property or method, and have it whisk you away to the correct msdn entry for the item of interest. When using third party libraries, it is great when they are able to provide you with the same ability to search API documentation from within Visual Studio. This is all possible, but I recently discovered, not necessarily trivial, and there is a really big gotcha if you want to add it to your Continuous Integration process.

To start with you need to ensure that all developers of your API are meticulous about putting XML Summary comments on ALL public facing entities. This is a good practice whether or not you intend to distribute the API.

The second step is to use a tool like sandcastle to build compile the xml comments into a usable format. Sandcastle has a number of different outputs including the classic .chm (Compressed html) output that is easy to read and can be distributed with your project, however, if visual studio integration is what you are really after, then you need the Html2 help output (*.hxs). As there are a lot of different settings for sandcastle, it is good to use the SandCastle Help File Builder that provides a nice settings based interface as well as a command line utility to aid you in building your help files.

The third step is to create an installation package (.msi or .msm) to distribute your documentation to the millions of developers who will be using your API to write the next killer application. There are a few ways of achieving this, to do this from scratch you could follow these intructions, however, in the Visual Studio 2008 SDK, there is a project type called the Help Integration Wizard that attempts to automate this process for you. This very simple project template works great right up to the point where you try to integrate it into your Continuous Integration Process.

The first problem is that the solution created contains 2 vdproj files, which means that you can’t build them with MSBuild, and are forced to pollute your build server with Visual Studio and compile it using the devenv command line. However, even after you have taken this hit, the pain doesn’t stop there.

The reason is because of 2 files that are included surreptitiously in the project path but not added to the solution. The executable FixRegTables.exe, and a merge module called MSHelp2_RegTables__RTL_---_---.msm get added to the CollectionFiles folder for the merge module project that is created. If you fail to check these in to your source control solution, your CI build will break with an error about a post build step. However, if you do add them to your source control repository, then chances are when you perform a “Get Latest” operation, these files may well be marked as “Read Only” as this is standard practice for many source control systems to ensure that source files are not modified during a build. In this case, the build will work as expected and produce an msi file, however, the resultant msi file will not install anything when run. The issue is that the Post Build step mentioned above needs to modify the merge module.

The solution is to either check the 2 files in to your source control system, and make the merge module writable as part of a pre build step, or to copy the two files from their location in the visual studio sdk into the Collection Files folder as a prebuild step.

Technorati Tags: Visual Studio 2008,SDK,Help Integration,HTMLHelp2,Continuous Integration

Bragging time

2009-11-08T18:34:00.001+11:00

On Friday, I successfully passed Microsoft Certification exam 70-561. This now makes me a MCTS: .Net 3.5, ADO .Net Applications.

I am going to do a few more exams over the coming year, but I’ve yet to decide what the next one will be.

One small gripe about my laptop

2009-11-07T16:22:00.001+11:00

I absolutely love my new laptop, especially since I put an SSD in it. I have one small gripe however, and that is with the keyboard, or rather the lack of one key that I have become very reliant on, that is the “Right Mouse Key”.

I am often too lazy to plug in my mouse, or often (like now) not in a position to do so (reclining in a comfortable chair with my Feet up, and the laptop on my lap), and I have become quite used to the keyboard shortcuts for all the applications I use, which is why I have always loved the right mouse key, but alas on the Dell Precision M4400 laptop, no such key exists.

Having said that, just in doing my research for this rant, I have in fact discovered that Shift+F10 does the same thing, problem solved! Still would be nice to have a single button for this though.

Laptops, SSD’s, vhd’s and Hyper-V

2009-10-05T23:40:00.001+11:00

I recently splurged and purchased one of these for my new laptop. I was finding hard to justify the extra expense for the 128GB over a 64GB SSD, knowing full well, that if I was careful I could run an OS and all the applications I normally use on 64GB, but I thought… what the hell. This weekend I discovered the “boot from vhd” feature of Windows 7, and I can say I am so glad I spent the extra money.

I have successfully created a dual boot for Windows Server 2008 R2 running on a 25GB vhd stored on my SSD. It is a bit of work, mucking around with DISKPART and BCDEDIT, but once it’s set up, it just works a treat, and thanks to my SSD, the speed is awesome. My main reason to do this was to play with Hyper-V as I have been charged with setting up a build server and test server at the client I am currently engaged to.

New Laptop

2009-05-16T16:45:00.001+10:00

having recently gone back to work after returning from a year in Buenos Aires, I have been tooled up. One of those tools as you may already have seen from my previous post is a new laptop, and just by way of showing off, I thought I’d post the specs up.

Mobile Workstation AW-M4400n - Dell Precision M4400n-series Base
Intel(R) Core(TM)2 Duo Processor T98002.93GHz, 6M L2 Cache, 1066MHz FSB
8GB (2x4GB) 800MHz DDR2 SDRAM
15.4 " Ultrasharp(TM) WUXGA (1920x1200) RGBLED Display with TrueLife(TM)
NVIDIA Quadro FX 770M, 512MB dedicated memory
8X DVD + /-RW Drive
2nd Hard Drive 120GB SATA (5400RPM) Hard Drive (External)
Integrated 0.3 Megapixel VGA Webcam with Single Digital Mic for WUXGA LCD
Wireless Network Cards -Intel WiFi Link 5300 (802.11 a/g/n 3X3) MiniCard

I have also made the decision (as many other Readify people have) to install Windows 7 RC on it. So far there have been no serious problems, but there are a few things I have found and I will blog about them as I find and (hopefully) fix them.

The first one was Google Chrome not working, but there are many posts about this, however even after this, Chrome does seem to be a little unstable from time to time, the end result of which is I end up spending half my time in IE8 which I have no real problem with.

The second problem was the bluetooth issue I talked about in my previous post.

I do get a little bit of instability from Visual Studio 2008 from time to time. I think it has something to do with Team Member Presence information in VSTS Team Explorer, and so I turned off this feature and it now seems to be a lot more stable.

Technorati Tags: Windows 7 RC,laptop

Windows 7 and Bluetooth Issues on DELL M4400

2009-05-16T13:44:00.001+10:00

Since yesterday evening I have been trying to get Internet connection sharing happening between my new brand HTC Touch HD, and me brand new Dell Precision M4400 running Windows 7 RC.

as could be expected, there were some issues. In fact, I still haven’t managed to get it working, but I am 1 step closer to a solution. The first and most overriding problem was that the device drivers on my DELL weren’t working properly. A quick google search and a few clicks later, and I found this discussion on notebookreview.com from someone having the same problem as me with their M6400.

http://forum.notebookreview.com/showthread.php?t=340326

So I tried it, and sure enough it worked. I decided to do the right thing by the community and reply to the thread, thanking them and adding the DELL M4400 to the list of laptops that it works with, as well as adding a link to the right driver downloads for the M4400, but alas, even after registering with the website, it appears as though I don’t have the necessary privileges to reply to the forum. So I will disseminate the information here on my blog.

So if you have a DELL Precision M4400 and are trying to get your bluetooth to work, you need to

Download these drivers from DELL
Open the folder where the downloaded executable resides
Right Click and select “Troubleshoot Compatibility”
Select “Troubleshoot Program”
Select “The Program worked in earlier versions of windows, but won’t install or run now” and Click “Next”
Select “Windows Vista” from the top of the list and select “Next”
Click “Start the Program”

At this point the bluetooth drivers should install no problems.

I have now successfully created a bluetooth connection between my phone and my PC, and can happily use some of the bluetooth networking features. Unfortunately, Active Sync doesn’t quite seem to connect properly over bluetooth, I think it’s a problem with “Windows Mobile Device Center” on Windows 7 RC, and of course I still haven’t been able to get Internet connection sharing working. Any help or suggestions would be greatly appreciated.

Technorati Tags: bluetooth,Windows 7 RC,Internet Connection Sharing,DELL

New phone, was happy, then… suddenly… not

2009-05-14T23:50:00.001+10:00

A few days ago I was the happy owner of a new HTC Touch HD phone which I have been happily playing with. Sure it might not be as trendy as the iPhone, but I was pretty impressed. It has a 5 Megapixel camera, as well as a lower res camera for video conferencing. It has WM 6.1, plus HTC’s own Touch Flo home screen which gives you some of the iPhone look and feel that Windows Mobile is seriously missing. I was starting to really enjoy it, downloading all the applications I wanted on it, connecting it up to my WiFi network and watching You-Tube videos on it, etc… then it came time to change my SIM card. I opened the back, took the battery out, replaced the SIM card, replaced the battery, and then this.

The phone would not boot, the OS would not load. I tried 20 different things, changing the SIM card back, doing a soft reset, without SIM card etc… nothing helped. The only thing that did eventually fix the problem was a hard reset. This means wiping ALL your data. So I am currently trying to re-install all the software I had up and running. Not very happy, starting to loose confidence in the device, and starting to wonder whether I should’ve purchased an iPhone.

CityRail not just biased towards iPhone

2009-03-07T00:32:00.001+11:00

It seems that a number of mobile application developers have fallen foul of CityRails timetable copyright antics. At one stage CityRail even went to the extent of scrambling their website in order to stop people screen scraping. I repeat, it may be true that they legally own the copyright to this information, but that is in no way helpful to the people who want (and dare I say have a right to) this information. It’s not that hard, really. Other governments have managed to do it, what is wrong with the NSW Government.

Government 2.0

2009-03-06T15:31:00.001+11:00

Reading an article in The Age about CityRail’s attempts to stifle a very useful iPhone application reminded me of a seminar I attended at Web Directions North 2008 on Government 2.0. The thrust of the seminar was basically for governments to encourage this sort of effort. They are the sacred holders of the information that the public need, they should release this information and let the general population use it in the way they see fit. Governments don’t always have the best ideas, or even if they have great ideas, they don’t always have the necessary resources to realise them. In the past forward thinking governments have found abundant creativity and resources if they simply provide the information and let the community do what is needed with it. Sure you may need to provide some disclaimers for legal purposes, something like “While <insert name of government agency here> supports development of third party applications based on the data we provide, we cannot vouch for the accuracy of the program, nor it’s suitability for any particular use” I’m sure every application developer would be happy to include this in there terms of service.

In this specific incident, CityRail has used the copyright argument saying we own the timetable information. Which is probably legally true, but at the same time totally unhelpful. People like Alvin Singh should be rewarded for there creativity and effort, not threatened. He has even offered to give CityRail the source code on which to build a better application. CityRail should take up his offer, or at the very least, bring out a competing application that is more accurate (ie includes the latest information about service disruptions, etc…), and has better features, that will quite naturally defeat Alvin Singh’s application.

Conficker and Password Advice

2009-01-24T01:22:00.002+11:00

I was just reading an article in The Age today about the Conficker virus, and the article was pretty straight forward… bad virus, infected lots of machines, uses brute force to crack passwords. Some of the advice that was given was obvious, use stronger passwords, make sure your OS is up to date, and any anti virus software you use is also up to date. However, the last sentence really bothered me

From the article :

"Go get a notebook, keep it next to your computer and record your password in it. No hacker in the world can hack the written page locked away in your office."

This is a quote from David Perry a “Software Security Specialist” from Trend Micro.

I have not heard worse advice regarding password security in many many years. Sure a virus can not leap out of the computer file through your notebook and try everything written down, but what about anyone who uses your office. Your cleaner, your spouse, your kids, their friends etc…. It’s the same basic principle with you bank card PIN number. If you write it down, then another person can potentially see it and use it, and if you follow the other advice in the article about using numbers and special characters (which you should) it makes it quite easy for a human to pick out a password from a heap of other random scribbling on your desk-side notepad. You may well say, but it’s only my spouse and my kids that use my office, and I trust them. Sure, but I’ve never seen it as an issue of trust in these cases, I see it as an issue of protection. I trust my spouse implicitly, but will never tell her my password. The reason being that if there is ever a security violation using my account, and my company asks me who else may have been able to find out my password, I can honestly and confidently say no-one, not even my spouse, there-by protecting her from any suspicion.

So what’s the solution. We know for a fact that username and password is a flawed security model, and we often talk about three-factor security, ie username, password and hardware key like a smartcard, as being more desirable. However, the reality is that the vast majority of systems are still protected by the humble username and password (two-factor security), so how do you choose a password that will protect your computer without the dreaded fear of forgetting it?

My tips are as follows. Pick something from your life that is obscure enough that other people wouldn’t be able to guess. As an example you may be into 70’s rock music, and in particular you may love the album from WHAM called “Make it Big”. That’s pretty obscure (not to say quite sad), or you may have just finished reading Dostoyevsky’s “The Brothers Karamazov” which is in itself an achievement worth remembering long after. Pick something from this aspect of your life, a phrase or a name, so in the first instance you might pick “Guilty feet have got no rhythm”, in the second case lets say one of the characters names “Dimitry Fyodorovich”, although a minor character may work even better, now adapt this to a password. Leave out some words and leave out spaces (if you want), then replace certain letters with numbers, capitalize others, and replace others with special characters like this “gui1tY#eeTn0rhythM”. In the second case something like this might would work “d!m!trYfY0d0r0v!ch”. Make sure the password length is greater than say twelve characters, and satisfies any other constraints your Systems Administrators want to put on your corporate network. Now you have a password that is, for you, fairly easy to remember without the necessity of writing it down, and extremely hard for a brute force crack to guess, or (and this is where the obscure thing comes in), no one who knows you will be able to guess either.

The other good thing is you now have a scheme for creating really interesting passwords, and it means that you can change the password regularly, and each time it is a game to see just how creative you can get in picking your new password. The more creative you get, the harder it will be for anyone or anything to break it, and the easier it will be for you to remember what it is.

Technorati tags: Security

WCF error when transferring large byte arrays

2008-10-30T03:55:00.003+11:00

Today I was receiving the following error when attempting to send a reasonably large byte array to a WCF Service.

There was an error writing to the pipe: Unrecognized error 232 (0xe8).

As you can see another potential candidate for least descriptive error message known to mankind. The only references to this error that google could find (here and here) were not all that helpful as they were about timeouts and closing the channel, in my case nothing was timing out. The solution was eventually obvious, a simple oversight on my part, the maxReceivedMessageSize property of the binding needed to be set to an appropriately high number so that the channel would allow for such a large message to be sent. Note you'll also need to set the maxArrayLength parameter of your readerQuotas configuration entry to a suitably large number otherwise it won't accept your byte array. eg.


<binding name="MyBinding" maxReceivedMessageSize="99999998">
  <readerQuotas 
    maxStringContentLength="65536" 
    maxBytesPerRead="9000000"
    maxDepth="96" 
    maxArrayLength="2097152"/>
</binding>

Personal Projects

2008-10-10T03:15:00.002+11:00

Most developers have them, an idea in the back of their mind for a project they'd like to write. Could be anything from a little utility app to automate something that they find tedious, through to the next big web 2.0/3.0 application that will change the world (or at least make them very rich). I know I always have a few that I've got on the boil from time to time. Mitch Denny has even gone as far as posting up some of his ideas. My biggest problem has always been finding the time between a fairly full on job, and a reasonably active social life, to sit down and make some significant progress on any of them. This in fact has formed part of my justification for taking a year off work and travelling to Argentina (for a full list of justifications see this post on our tangotrails blog).

I have been making some significant progress on a couple of my ideas over the past few months, and it has re-affirmed to me the professional value of having personal projects like these. The main project I've been working on is an idea to combine my technical side and my creative side. I won't say any more in this post, except that I want to see how far the web can push the way we experience the narrative form.

Because it is my project, and my idea I have full ownership over everything, good and bad. I have full ownership of the codebase, I have full control over the direction, over deadlines and milestones, I have full responsibility for testing and bugs, I'm even in charge of running a beta program (which I intend to do soon), and marketing the product. Legal issues, hosting provider, selection of tools, graphic art, everything is my responsibility, and mine alone. It has been a really educational experience to work on a project so fully from end to end, and not be able to assign parts of the application off that are "not my problem" to someone else. I am learning so much, not just about new technologies, but also some of the non-technical aspects of getting a project like this off the ground.

I think forward thinking companies should try to encourage these personal projects for their technical staff, even if the projects their developers want to work on don't wholly align with their business goals. I'm thinking along the lines of a company integrating personal project time into their staffs professional development plan. Something like allocating X days of your professional development time to a personal project. Of course you may need a project approval process, but this shouldn't be aimed at the business determining if the project is aligned with their goals, but more that the project is significantly challenging for the developer for her to really gain something out of it. Also, a short presentation of their work, in whatever state it is in after the allotted X days, would ensure the developer is kept honest, and that information is disseminated to other developers in the organisation.

I can see this benefiting this hypothetical forward looking company in a number of ways. For starters, I've seen a lot of developers get bored with what their business wants them to do on a day to day basis. Often developers end up in roles where the domain holds little or no interest for them, or if it once did, the constant focus on it destroys some of the enjoyment they once had. Focusing, even for a short time, on something completely different, that they are really interested in, will re-invigorate the developer. Also, having developers play with technologies outside of their normal sphere of day to day experience in the business, will make them a valuable asset if the business ever needs to consider any of those technologies in the future. Being able to advertise this approach to professional development would also attract the right sort of candidates when recruiting. A really forward thinking business may also be open to the possibility that one of their developers might just be creating the next big web 2.0/3.0 application and might even be willing to partner with them in an endeavour to commercialise it.

Technorati tags: Development, Professional Development, software

XHTML 1.1 strictly ASP.Net

2008-10-05T00:40:00.004+10:00

For those of you who are interested in web standards, you may have noticed that when you create an ASP.Net application the standard project item templates create a web page that complies to the XHTML 1.0 Transitional doctype. This may or may not be what you want. XHTML 1.0 Transitional is for existing websites that are in the process of migrating, or if a website specifically needs to support browsers that don't know about CSS. This is because XHTML Transitional still supports the presentation based tags such as <font> and <center>. New websites however, should generally adhere to the XHTML 1.1 strict doctype which does a better job of enforcing separation of content from presentation. So as an ASP.Net developer I have to re-jig all my pages when I add them into my solution, to comply with this doctype.

No longer do I have to do all this manual work, thanks to a fellow Readify colleague, Damian Edwards, who's work I really admire and respect, you can now add proper XHTML 1.1 strict templates when developing ASP.Net applications. Damian has created this CodePlex project that installs XHTML compliant item templates for ASP.Net Web Project items such as Web Form, Master Page, Ajax Web Form, Ajax Master Page, HTML Page, and StyleSheet for Visual Studio 2008.

Note : you also have to add the following to your web.config

<xhtmlConformance mode="Strict"/>

This tells the ASP.Net rendering engine... "no I really meant it when I said strict, and I know what I'm talking about."

What does this guarantee you as an ASP.Net developer?

Because of ASP.Net's browser capabilities detection mechanism, what this guarantees you is that when ASP.Net is serving a page to a modern browser, it will serve XHTML 1.1 strict compliant markup. Be careful, this does not necessarily mean that your site will validate with the W3C Validation Service, in fact I've been caught out on this one before.

This reminds me that I've been meaning for a long time now, to write a more concise article on developing standards compliant web pages in ASP.Net, I'll put it on my daily goals list, item #44 right after "learn Spanish". (sorry for those who don't quite get the obscure red dwarf reference).

Developing Live Writer Plugins

2008-09-04T09:09:00.002+10:00

I really like Live Writer. I have been using it for some time now to maintain a number of blogs, and I think the advantages of this offline blog post editing tool outweigh the advantages of the online equivalent. I'm not going to go into an in depth feature comparison here, but I will say that one thing I really love is the ability to have plugins. I have installed a number of plugins myself, the two that I use most frequently are Flickr4Writer and Tag4Writer. They are just simple little utilities that generate small bits of html and save you from the drudgery of hand crafting cookie cutter HTML yourself.

What I love more about live writer plugins as a .Net developer, is that it's fairly easy to develop them yourself. There are some fairly good articles on this that I've found, so for anyone who wants to get started writing their own Live Writer Plugins, here is a list of recommended reading.

Developing plugins for Windows Live Writer by Scøtt Lovegrove
Writing Plugins for Windows Live Writer - Getting Started
and of course there's the MSDN documentation msdn documentation

These are all great resources, but the one thing that each one of them fails to mention is that if you are developing on vista with UAC enabled (as you should be), or if you're developing on Windows XP as a least privileged user you need to change the permissions on the C:\Program Files\Windows Live\Writer\Plugins folder to allow the user you are running visual studio under to write and modify dll's in that directory.

Technorati tags: live writer, development, Visual Studio, plugins, .Net

Pocket PC development connectivity issues

2008-07-26T10:08:00.002+10:00

I have been returning to my roots and doing some Pocket PC development recently. I haven't really done that much since Visual Studio 2003, and to be honest, apart from SQL CE 3.5 which is a modest improvement on SQL CE back then, I feel like everything is pretty much the same. Emulators are slow, debugging is tedious, and device connectivity is sporadic at best. I was happily developing away when for no apparent reason I got the following error when trying to connect to my device which was happily Active Syncing,

Connection failed. A socket operation was attempted to an unreachable network.

I tried everything, disconnect, re-connect the device, shut down, restart Visual Studio 2008, even rebooting my laptop, I was still getting this error. Google was no help either, it appears to be a fairly generic, but seldom encountered winsock error not specifically related to Pocket PC development. What eventually solved my problem was performing a soft reset on the device.

Stopping applications that run automatically on startup

2008-07-23T06:55:00.002+10:00

I thought I'd put this information here mainly for my own purposes, because every time I decide to clean up the various applications that run up automatically when I can never remember the windows system tool to use to view these, so I end up doing a google search and have to wade through all the commercial products that do the same thing to find the name of it.

It is mscofig.exe, and is located in the %SystemRoot%\system32\ folder. On Vista you can just click on Start then type msconfig and hit enter. It is also accessible through the Administrative Tools folder in Control panel.

Once this has opened you can go to the startup tab and deselect any annoying programs that appear in the system tray that you really don't use, and don't provide an option for you to stop this behaviour.

Technorati tags: msconfig, startup, Vista, system tray

Consultingware model thwart with inefficiencies

2008-05-06T14:31:00.002+10:00

A large proportion of my career has been spent writing consultingware, and I've come to the conclusion that consultingware is inherently an unstable model full of inefficiencies and road blocks that can lead to an un-productive environment.

Let me explain the consultingware model. Consultingware is where a software house (lets call them the supplier) creates a package that satisfies 80% of a generic market niche, and then goes about selling to customers in that market, the base package plus consulting services to customise the remaining 20% to deliver a final product that the client requires.

This sounds fair enough, especially considering that there are many systems that would seem very similar, but because of the way the business is run, just require some tweaking at the end user level to make it fit straight into there current business processes. It is also attractive from the point of view of the supplier producing the consultingware, because although it's not the ideal model of selling bits of paper with license numbers on them (i.e. shrink wrapped software), if you find the right niche, you can charge big bucks for the initial package to each of your clients, and then continue to charge fairly lucrative consulting rates to customise the application because the client is essentially locked in to buying consulting services from the one supplier. However there are a number of inefficiencies and false economies that are inherent in this model if either the supplier or the client are not careful.

Let me start with the ongoing issue of code maintenance and improvement. Usually the supplier has spent a long time producing this base package with little or no financial support from any actual sales, and unless you have an absolutely brilliant development team there are always going to be code maintainability issues, performance issues, architectural problems, scalability concerns etc.... The shrink wrapped and internal development models have fairly well known processes for continuous improvement of code. The issue with consultingware is two fold.

Firstly the client is never interested in things they can't see. They are only ever interested in high level features that they can use in their day to day business. Even if they start to complain about things like performance, they are likely to be reluctant to fully fund the necessary development time to fix these issues considering they are only one of potentially many clients (current and future), who will see the benefit of these improvements. The same logic can also be applied to bug fixes, why should client A pay consulting rates for a bug to be fixed just because they were the first to see it and demand it be fixed. This leaves the supplier with the responsibility for these kinds of structural issues which creates a bit of a dilemma.

Inevitably if these kinds of improvements are done without direct funding from a client, this means that developers who could normally be charged out at the lucrative consulting rate become non-chargeable for a time loosing the supplier money. Not only that but if the changes actually improve developer productivity, (i.e. it makes adding features more efficient), the supplier will start to see a negative shift in their profit/feature ratio as developers start finishing chargeable features quicker.

Admittedly this is a false way to look at the problem. Ideally the way the supplier should look at it is that improving their own efficiency will make their clients more profitable and more likely to expand their business and hopefully require more of the suppliers consulting services as they grow. This is sometimes a very difficult argument to propose to bean counters when it is full of if's and maybe's and things that can't be measured easily.

The next problem is the way in which suppliers attempt to grow their product. They think that they will use their clients consulting dollars to add generic features to the product so that they can build a more out of the box solution for the next client (i.e. turn that 80% functionality into a 85% or even 90%) so that they can get more sales. This is problematic because the client again is really only going to be willing to pay for what they actually get benefit from, and to implement a feature generically so that all clients can use it, will inevitably cost significantly more than implementing it specifically the way the client wishes to do it. For example if Client A has a SOE that dictates everyone uses IE7, they aren't going to be interested in paying the supplier to ensure that the software works with Firefox, safari, opera, etc...

Another issue that frustrates the problem is the upgrade cycle. As more clients come on board, and more features are added, when clients upgrade, they take on risks associated with features they never asked for. This requires a very modular approach to adding features, but even still often changes to the core software are required to add new features, and this will impact on existing features.

This feeds into the final problem I want to discuss, and that is who is responsible for testing? I've seen a number of wrong approaches to this, including "The client will test it". I would suggest that any client that takes on a piece of consultingware needs their head read if they don't do some form of user acceptance testing. However, again, the client will really only be interested in testing the features that matter to them, and not being a software company, they won't really appreciate proper regression test cycles. Ultimately a proper regression test and defect fixing cycle is going to be left to the supplier, but if no client is willing money towards that cycle, it will very quickly eat into the "lucrative" consulting rate the supplier charges for additional features.

So I guess I've really just stated the problem here, and offered no real solutions, and guess what... I don't intend to... at least not in this post. I would like to hear how other people have seen these issue tackled from both sides of the fence (client and supplier), and in a few weeks time, I'll put together another post with what I think are some potential solutions to the problem.

Tango Trails

2008-04-06T09:32:00.002+10:00

Those who know me, know that I am a passionate Argentinean Tango dancer. Those who know anything about Argentinean Tango know that if your serious enough about Tango, Buenos Aires becomes like Mecca... you have to visit it once in your life. Niki and I are planning such a trip, and as such have decided to start a blog about not only the trip, but anything relating to our journey into Tango. For anyone that's interested, the blog is called "Tango Trails".

Problems with Sony Camcorder software

2008-03-30T19:36:00.002+11:00

I purchased a Sony HDR SR7E camcorder a while back, and for the most part I love it, but there is one thing that is really frustrating me. The software that comes with it to playback videos on your home PC in the native AVCHD format (Motion Picture Browser) has a really unusual problem.

As you can see the top half of the screen is fine, but the bottom half is just a solid green line. The camcorder itself plays back the videos fine, and when you make a DVD or convert the videos to mpeg-2 format, the resultant video is fine, the problem only seems to exist in the Motion Picture Browser software.

I am running Windows Vista, Motion Picture Browser Version 2.1.00.04170 on a Dell Latitude D820 with 2GB RAM. I can't seem to find any mention of it on Sony's website, the only other mention of it I have been able to find is here. If anyone else has seen this issue, or if anyone else knows what the solution might be, I'd love to hear about it.

Technorati tags: Sony, handycam, HDR SR7E, AVCHD

Why Web Standards?

2008-01-29T17:13:00.001+11:00

I am currently in Vancouver attending the web directions north conference. So far it's been really good, and I have some great ideas for some blog posts on some cool CSS techniques I've learnt from Andy Clarke's workshop on CSS, but you'll have to wait as I want to play with some ideas for a while. For now I want to talk about something else.

I have asked John Allsopp (the man behind the company behind web directions) for a video interview tomorrow, and he's agreed. Amongst other things i want to ask him the question "As asp.net developers, why should we be worried about web standards?" (this question is mainly to let him evangelise about web standards), and i thought, maybe I should attempt to answer that question myself, so here goes.
If you are writing a public facing website (or web application as is most common for asp.net developers), then the truth of the matter is that you CANNOT assume a particular browser. Ie7 DOES NOT, & WILL NOT be the only browser that you have to consider. Firefox is an extremely popular browser & looks like maintaining (if not increasing) it's popularity, i even have it as my default browser, not to mention opera, safari, and others.
Further to this, you CAN NOT even assume windows xp or vista as your base operating system. there are many people who are still running windows98 (mainly for licensing reasons, but they still represent a market you may want to reach). Plus the hordes of people who still haven't had the Microsoft chip implanted into their neural system, and insist on running Linux, mac OSX, etc. Add to this the growing market of handheld devices such as PDA's and phones, all with their different browsers, and you have a truly overwhelming number of combinations and permutations that are possible.

Now lets throw into the mix something that I have seen very little support for in the ASP.Net community as a whole, and that is Accessibility. When most people think of accessibility, they think of screen readers, but that is just one end of the spectrum, people with impaired vision (my partner included) often simply want to increase the font size a little, which renders a lot of designs I've seen in the past quite bizarrely to say the least. Why should we worry about Accessibility? I hear you ask... forgetting the ethical concerns, and the fact that the web was originally designed to be accessible to ALL, the fact of the matter is that (at least in Australia) you can be SUED if you do not comply to accessibility standards, the Sydney Olympics was sued to the tune of $500,000 for this very reason.

A quick concession here is that Web Standards compliance does not always mean accessible, there are many things you can do and still remain standards compliant, but be completely inaccessible, the issues for accessibility are somewhat more ethereal, however, web standards is a fairly good starting point.
Given all this, as a developer, you still need to pick a browser support matrix, otherwise you will go insane attempting to test with ALL versions of ALL browsers that have ever been in existence, (I'm not sure if anyone still ever uses lynx, the good old text based unix web browser, but it is certainly and interesting exercise to open up your website in it and see what happens), but even once you have determined your browser support matrix, unless you have a very narrow focus group, this will include more than just IE7. Also even though you might not be directly considering older browsers, it doesn't hurt not to purposely break them just because you can.

Having decided on a browser support matrix, then you will quickly realise that the best way to not only get the best results in all your current browsers, but also to anticipate V.next of those browsers is to use to the best of your ability the currently agreed upon web standards.

For some reason this seems to many ASP.Net developers to be a real problem, this is partly because of some cultural issues with developers in the Microsoft space, and partly because of the way ASP.Net abstracts the developer from the final markup that is actually rendered to the browser (I want to deal with this at length in another post), but to be honest, there is really no excuse. There are very few effects that you would like to achieve that you cannot do through the use of standards compliant markup and css, and a little bit of fore thought. If you go the extra mile and approach it from a more ideological point of view where you attempt to separate the content from you presentation, you get the added benefit of making you website at least readable on all different types of browser, not to mention generally a lot closer to being accessible by assistive technologies (such as screen readers) the likes of which most of us will NEVER get the opportunity to test against.

Of course if you are working on an Intranet site for a company that runs an SOE that they will NEVER change (eg Windows XP with IE6), then feel free to write whatever browser specific functionality you need to to get the job done, and pray they never upgrade to IE7.

Technorati tags: WDN08

Heading off to Web Directions North

2008-01-27T12:00:00.001+11:00

I am currently sitting in Melbourne Airport about to board a plane for Vancouver to attend the Web Directions North conference. I am quite excited about the people that are speaking there, and I am hoping to gain some great insight into aspects of design and user experience as well as informed opinions on the future of new web technologies. I have also been asked to run a "Birds of the Feather session" on aspects of Internet security and Identity. I am certainly no expert on either of these, but my interest has been sparked by recent experiences, and I look forward to discussing these issues with people who know a lot more about it than I do.

Technorati tags: WDN08

CAPTCHA foiled by Mechanical Turk

2007-11-28T15:27:00.001+11:00

An article on CGI Security about CAPTCHA highlights a very obvious flaw in the whole philosophy of CAPTCHA. For those who are too lazy to click the link and read the 2 paragraphs the article basically describes how some very inventive worm writers enlist the help unknowing participants who are visiting pornographic web sites to play a game that offers to reveal a pornographic image piece by piece as the user breaks a code (obviously the CAPTCHA the worm is currently observing at the site it is trying to create a bogus account for).

I have to concede this is a very clever use of the Mechanical Turk idea. CAPTCHA is designed to weed out humans who are genuinely trying to sign up to a site and create accounts from computer based worms that are trying to create bogus accounts for their writers evil ends. CAPTCHA actually stands for Completely Automated Public Turing test to tell Computers and Humans Apart, and there is the flaw... the Turing test is designed to distinguish between humans and computers, if you the computer you're testing can enlist on-mass large numbers of humans to take the test for it, then the test will be rendered redundant.

This raises some very interesting concerns for identity verification. Everyone wants their site to have the smallest possible barrier to entry, even the simplicity of CAPTCHA systems can sometimes prove to be a barrier eliminating a small number of false positives. So what's the alternative? I'm not quite sure, maybe something like Windows CardSpace, or a competing identity management protocol that allows a third party company trusted by both the provider and the client to validate credentials, but of course there would have to be overwhelming take up of the same protocol set by both providers and clients which seems to me to reduce to a chicken and egg kind of problem. Also the cost of maintaining this systems integrity by the trusted third party would need to recouped from either the client or the provider. I have to confess, I'm not really an expert on identity management, so I'd love to hear others thoughts.

Web Directions North

2007-11-28T11:15:00.001+11:00

Well, it's official, I'm going to Web Directions North 2008. Last year I went to Web Directions South, and really enjoyed it, this year I was in Cyprus during Web Directions South, so I'll have to settle for Vancouver. It's a real pity because I would have loved to have heard Chris Wilson (IE Platform architect at Microsoft) presenting on Moving the web forward. I must say, he is a brave man as a lot of people in the crowd would not exactly have been Microsoft fans, and would blame IE's standards issues for making their jobs harder, but I think his slide deck, and his audio interview with Kevin Yank from Sitepoint, state the dilemma fairly well.

So what am I looking forward to at Web Directions North?

I decided that considering I am travelling so far for this conference, I should probably attend the workshops, so I've booked myself in for Transcending CSS, and Desgining really usable websites. As a developer I feel that one of my weaknesses is web design, and as such I think that these workshops will be hard, but rewarding.

As far as the conference sessions go, all of them sound really interesting, but I'll just highlight a few that I am really excited about. Being interested in security, the Ajax Security session is looking good, it will be interesting to see Silverlight and Adobe Air compared, given my background in PDA development, I am interested to see where Mobile Web Design & Development is going to go, and given the presenters (Dave Shae and John Allsop), Where's your web at should be very interesting indeed.

Technorati tags: WDN08

Speaking on Visual Studio 2008

2007-11-28T08:57:00.001+11:00

I have become a big subscriber to the Chris Hewitt school of self education. I have been working with Chris at Readify for bout 3 years now, and I remember back when I first joined he told me "If you want to learn a new technology, simply book yourself in to do a presentation on it in 2 months time". It is so true, the fear of making a complete fool of yourself in front of your peers is ample encouragement to get yourself across any topic. I have done this a few times now with technologies like Ajax, Vista, and SQL Server. I am now doing it for Visual Studio 2008 which has just RTM'ed. I have been playing with the beta versions of VS2008 for a while now, and today I am now installing the RTM version. I am speaking at Victoria .Net SIG in December. There are about 5 speakers doing 20 minute segments on "What's new in Visual Studio 2008 ...", I am doing "What's new in Visual Studio 2008 IDE".

Check the Victoria .Net SIG web site closer to the date for more details, both the date and place are still up in the air at the moment.

ASP.Net and Standards (cont.)

2007-11-27T12:59:00.000+11:00

My collegue Damian Edwards pointed me to an article on MSDN (ASP.Net and XML) that explains the behaviour I described in my previous post on Asp.Net and Web Standards . From the article...

"If you submit an ASP.NET Web page to a validation service such as the W3C Markup Validation Service, ASP.NET might render a version of the page that does not conform to XHTML standards. This is because the validator service does not report itself as a browser type that ASP.NET recognizes, such as Internet Explorer or Mozilla. When ASP.NET cannot recognize the browser type, it defaults to rendering downlevel markup, which does not include XHTML-conformant elements and attributes, or features such as cascading style sheet styles."

The article also discusses how to configure ASP.Net using a browser capabilities file to force it to render valid XHTML. I'll have a play with this tonight when I get home.

Cross Site Scripting Detection Tool

2007-11-27T09:48:00.000+11:00

Just discovered a very interesting tool from Microsoft to combat Cross Site Scripting Vulnerabilities.

The XSS Detect Code Analysis tool has been released in beta, and does static code analysis to determine potential XSS Vulnerabilities within ASP.Net applications. I ran it on my standard XSS test application and managed to detect the standard XSS mistakes. I will say this though, it is still in beta, and when I tried to run it over our real site, the tool managed to crash Visual Studio. I can't wait til it RTM's, I'll be following this tool with a lot of interest.

ASP.Net and Web Standards

2007-11-24T00:06:00.001+11:00

We were having a discussion the other day on our internal tech list about Web Standards and validation, and I made the comment that I like to treat web standards validation like compiler warnings, in production code ideally you should not have any, because even though the vast majority of them are benign, and browsers will happily ignore them, they could be hiding 1 or 2 that are going to cause you grief. A colleague of mine Darren Neimke, challenged me to show any of my production code that I had that would stand up to this test. So I hurriedly tried to search around for something and ended up sending him a link to my, very much work in progerss site that I set up to play around with some ajax stuff scottbalwin.com.au, just to show him that I do try to put my money where my mouth is. Of course I did double check to see if the front page validated, in fact I checked a few of the links, and all that I checked validated fine. What I hadn't quite expected was for Darren to take the time to go through all of my links until he found one that didn't. On one of my ajax playground pages I had 6 validation errors. Ok, my bad, so what was I doing wrong? Well it turns out that 2 of those errors were due to me using a code syntax highlighting control, that in and of itself produces standards compliant html, but when you wrap it inside a <pre> tag, as it suggests in the sample that comes with it, you get an error saying <pre><div> aren't aloud to go together in xhtml transitional. Ok, simply change the <pre> to a <div style="white-space : pre"> and the problem goes away with no loss of formatting... Yay, but what about the other 4 errors?

What was really weird is that when I did my usual test of openning it up in IE from my dev environment, viewing the source and then pasting the source into the validator, it validated perfectly. So confident that I'd fixed ALL the issues I uploaded it to my website. I then used the validator to test the url, and lo and behold 4 errors... doh! it took me a while to figure out what was going on, but eventually after realising that some controls in ASP.Net can emit different html depending on the user agent that is requesting the page, and a quick little test later on my own machine and sure enough, if the user agent is IE (or firefox for that matter), the calendar control emits perfectly standards compliant xhtml transitional code, but for reasons best known to a small group of developers at Microsoft, if the user agent is "Unknown" or whatever the W3C validator claims as the user agent, it emits non standards compliant html. So I am currently trying to figure out the best way to fix this issue so that even unknown user agents receive valid html.

I now understand why people who are really serious about standards often end up writing their own ASP.Net in place of the standard Microsoft ones.

Oh and by the way Darren, don't bother checking to see if my blog validates, because it doesn't, and it is simply due to laziness that I have just selected a standard template from blogger,and haven't attempted to beat it into standards compliance. One of these days....

Vote below the line campaign (more info)

2007-11-20T21:30:00.001+11:00

I Found a really good article in The Age that explains just how important your preferences are.

Slow and steady fixes the bug

2007-11-16T09:30:00.000+11:00

I just recieved a notification from Microsoft that a bug I reported in the XML subsystem of SQL Server 2005 in March of 2006 has finally been fixed and will be shipped with the RTM (I assume SQL Server 2008 RTM). It took them a while, but they eventually got onto it. It was an extremely obscure bug, but it seems as though I wasn't the only one to see it.
Unfortunately because I no longer work for that client any more, I will not be able to verify the fix, I guess that's part of being a consultant.

Ballarat .Net SIG

2007-11-15T08:32:00.001+11:00

Big thanks to Damian for inviting me to speak at the Ballarat .Net SIG, there was a small but responsive group of people a good venue, and of course Pizza and Soft drink, all the ingredients for a good user group meeting. I am always amazed at just how much you learn when you speak at these kinds of events.

Performing at the Hispanic Fiesta

2007-11-11T16:05:00.001+11:00

Niki & I are performing again, this time as part of the the Spanish Club's participation in the Hispanic Fiesta (otherwise known as the Johston Street fiesta or the Spanish festival).

Details

where : Hogar Espanol (The Spainsh Club) 59 - 61 Johnston Street Fitzroy

when : 3:30 pm, Saturday and Sunday the 17th and 18th of November 2007.

Vote below the line campaign (cont.)

2007-11-11T15:03:00.001+11:00

The group voting tickets for the 2007 Federal election are now available on-line at the AEC website, and I urge anyone who has read my previous post on the topic of voting below the line and is still thinking of voting above the line to download the group voting ticket for their own state to see just how their preferences are being re-distributed. Even if you are going to vote below the line, it is good to have a look at who different parties are preferring. As I take a quick look over the Victorian paper, I can't help but wonder at some of the deals that must have gone on under the hood, or even at the reason that some of the parties even exist in the first place.

Speaking in Ballarat

2007-11-09T08:43:00.001+11:00

I was just doing some research for my presentation next week at the Ballarat .Net Special Interest Group, and was trying to get some information on running Visual Studio 2008 on Vista (I usually only run Beta software on VPC's, so I have no first hand experience of this as yet), when I clicked on one of the links from my search only to find that it was a blog post by Charles Sterling advertising said event which reminded me that I should probably put in a plug for it on my blog, so that all two people who read my blog (yes that includes me), know what I'm up to. So here it is :

“Vista as a Software Development Platform”
with Scott Baldwin from Readify

“What’s new for Web Developers in Visual Studio 2008”
with Dave Glover from Microsoft Australia

When: Wednesday 14th November, 5:30pm for light dinner and drinks followed by presentations from 6:00pm to 6:30pm and 6:40pm to 7:40pm

Where: COMMANDER CENTER, BALLARAT, 1 Ripon Street North

I find it funny that the last time I did this very same talk back in June, it was also in partnership with Dave Glover, but last time he was talking about power shell, this time Visual Studio 2008 enhancements.

Technorati tags: Vista, Presentation, "Visual Studio 2008"

Evidence Based Scheduling

2007-10-30T10:51:00.000+11:00

I've always loved Joel Spolsky's common sense approach to the software development lifecycle, and his latest article on evidence based scheduling is no exception. It has a realistic approach to quantifying unpredictable elements of team development such as interruptions, meetings and the odd rebuild of your development environment. Estimating is always a big problem, and most developers (including myself) are usually quite optimistic about how long it will take for them to write a particular piece of code. This leads to an un-ending conflict between managers who want to know when something will be delivered, or how much a feature will cost to create. I think Joel offers a real practical approach to this dilema, and I am now trying to think of ways to integrate some of these ideas into a TFS project template.

The only thing i don't think Joel covered is how to go about predicting for the very first iteration, the very first time you start using EBS. I think that you probably need to pick a number or range of numbers to seed your velocity history. You probably need to be a bit pessimistic to begin with, but by the next iteration you'll have some better numbers to work with.

Vote below the line campaign

2007-10-27T12:13:00.001+10:00

Who do you prefer

Yes, it's that time again. The 2007 Australian Federal election campaign is in full swing, and for the vast majority of Australians it will come down to a choice between the boxer in red shorts in the right corner, John Winston Howard, and the boxer in the blue shorts in the ... other right corner, Kevin (I speak Mandarin) Rudd. Now I'm under no illusions here, although I am an active member of The Australian Greens, I realise that the next prime minister of Australia will be either Rampaging Rudd or Horrible Howard, however, there is the balance of power in the senate that is up for grabs, which is where I think the Greens can make a real difference, beyond the democrats failed attempts at "Keeping the bastards honest", I think The Greens can not only keep them honest, but also keep them on track on important issues such as climate change, social justice, transparency, and workplace relations. The subtle thing that a lot of people don't realise is that the only reason The Greens don't already have more influence in the senate is because of the above the line preferential voting system and the preference deals that go on behind the scenes that ordinary voters are completely oblivious to.

I have blogged about this before when the Family First Senator Steven Fielding was elected to the senate in 2004 purely on Labor preferences even though he received little more than one fifth of the primary votes that the Greens Lead senate Candidate David Risstrom received. All the Labor voters I know were appalled when they realised that they were responsible for this miscarriage of democracy, some have even vowed never to vote 1 for Labor again, others have decided to vote below the line and distribute their own preferences.

Now I know that The Greens are no strangers to preference deals, and there are some in the party like me who are frustrated when deals are done. I personally think that the whole preferential voting system needs overhauling, and have said this in the past at branch meetings. This issue is an entire blog post on its own, and in this post I want to concentrate on what can be done this election to get who we really prefer into the senate.

Now don't get me wrong, I don't resent Family First coming on to the political scenes, I think that one of the major problems with the Australian political climate is that their is a lack of representation, and Family First do represent a valid cross-section of Australian society who are entitled to their say. In fact my parents and my sister all support and campaign for Family First, it's just that Family First don't have the numbers and support to legitimately hold any seat in the senate in their own right. So what can we do?

I personally think that people need to stop voting above the line in the senate, and to analyze your how to vote cards for the lower house as well to determine if you really want your preferences to go the way your party of choice is suggesting. I concede that the preferential voting system is very difficult to understand, and it is sooooo much easier to put a single digit in the section above the line, but you need to realise that this gives the party you voted for the power to redistribute your preferences as they see fit. I don't trust any party, not even The Greens to redistribute my preferences. To aid with the understanding of the Preferential voting system, the Australian Electoral Commission has educational material, and if you are still thinking about voting above the line, then be sure to to check out the AEC's 2007 Election website closer to the election when they will post the Group Voting Tickets for the senate that will show how your preferences will be redistributed if you choose to vote above the line. It is instructional to see the AEC's 2004 election webiste, and in particular look at the Victorian Group Voting Ticket, and see how the Labor party preferenced Family First above The Greens.

I would like use this blog post to start a campaign to get as many people as possible to vote below the line in the forth coming election so that the behind the scenes wheeling and dealing over preferences is rendered meaningless. Please link to this post, or write your own blog post urging people to vote below the line, and let your preferences be known.

Technorati tags: election, politics, vote, prerefential voting, senate, preferences, The Greens, Australia, below the line campaign

Performing on the fringes

2007-10-06T23:49:00.001+10:00

Niki and I have been asked to perform some tango at a production by one of our close friends at this years Melbourne Fringe Festival. The show is called Movimientos, and there are some really great dancers involved, so it is worth checking out.

Technorati tags: tango, dance, fringe festival

On Holidays

2007-09-15T16:53:00.001+10:00

I'm currently on holidays in Cyprus. It is at this time that my non-geek blog tends to get a bit more love than my geek blog, although I do intend to finish off a couple of geek blog posts while on holidays, I will be writing about all my adventures on musings of a morbid mind.

Defense in Depth - (Part 2)

2007-09-04T09:06:00.001+10:00

In my previous post I described a three pronged approach to software security that is summed up by "Constrain, Reject and Sanitize". In this article, I'll discuss the "Constrain" part in some more detail.

Constrain

Developers are usually focussed on what their end users are going to want to do with the system, and this generally does not (in most cases at least) involve launching XSS or SQL Injection attacks against the system, in fact generally the target audience for most software know nothing of these things. So when the developer is designing a piece of software, they are looking at it from the perspective of a benign user who just wants the software to work. It goes without saying, however, that the benign user doesn't want their personal details divulged to hackers. So there comes a time when the developer of the system has to consider what a hacker might want to "inject" into their software. There is a general principle that ALL user input should be considered evil until proven otherwise. The first part of this process is to constrain ALL user input.

There are a number of ways that input can be constrained, based on the type of information you are expecting.

1. If the possible field values is a singl or multiple selection from a fairly small well defined set of values, only allow the user to choose from this set of values. This can be done using UI elements such as a list of Radio Buttons, a group of check boxes, a listbox or a drop down list.

2. If you require more freedom than this but the data has a strict pattern that you can check for, then ensure that you validate the entered data the user enters. . This can take a number of forms

a. Data of a particular type (ie Decimal / Integer / Date) should be attempted to be cast to that type as soon as possible and the user notified if the cast fails.

b. Valid ranges and lengths of all data should be enforced. i.e. an age field may be required to be > 18 but < 130, a name field may be 30 characters or smaller etc....

c. regular expressions should be used for things like email addresses, post codes, Tax File Numbers, etc...

This validation MUST at the very least occur on the server side. There is a trend to validate client side using javascript, and this can add a lot of difference to the responsiveness of your applicatoin, and even to the load on the server, however, it is a serious mistake to have only client side validation. In the first place the user may for various reasons have disabled javascript, in fact if you are being hacked and you are validating using javascript then the first thing the hacker will do is turn off javascript. Secondly, because of the nature of the web, you can never be sure exactly what you are talking to, it may say that it is IE in the Htttp headers, but in reality it could well be a program written by the hacker specifically for the purpose of fooling your application.

This kind of validation is called "white list" validation, because it looks at the problem from the point of view of a set of allowable formats for the input data, anything that does not satisfy this formatt is rejected outright. For example if you have a post code field, no one is going to be able to write any kind of attack that contains only 4 characters, all of which are numeric (0-9), similarly it is not possible to form an attack that looks enough like an email address to validate with a decent email checking regular expression.

The issue is though that some fields don't really lend themselves to this form of valdation. For instance description fields you generally want to be large text fields that can contain virtually any type of character. Even still, I would suggest you attempt to define a set of allowed characters and limit what the user can type into these fields. This article is not specific to web development, however, I do want to say something specific to web development. Some times you want to give your users the ability to enter rich html content. this is all well and good, but of couse it makes constraining the input quite difficult. You have to allow tags which means that you are potentially openning yourself up to Cross Site Scripting attacks. You might say that we can just reject any <script> tags, and we will get on to the rejection phase in my next post, but if you spend a bit of time looking at the XSS Cheat Sheet you'll very quickly realise that there are literally hundreds of ways to phrase an XSS attack. Also as the web is eveloving, and browsers implement the new standards (and new proprietary tags), the list of possible attack vectors grows without bound, so a site that may have been safe in ie5 days may without any extra work be vulnerable if the end user is using firefox 2.0 or IE7. This is a difficult problem, and one solution I have seen in the past is for the rich editor control to use its own format for storing the rich content. So the idea here is to have a format that allows for a supported subset of html. It is stored in the backend in this format and only transformed into html when it is needed to be rendered. This is still white list constraining and it works because the format that the control stores its data in will not have the necessary syntax to support tags that it doesn't know about.

The final thing I want to say is to give some idea of where this constraining should be done. The defence in depth paradigm requires that input be checked at every boundary. So the first boundary would be client side ie when the user first enters the data into the UI. Secondly the server side should validate the instant it recieves any data. From this point onwards, the data should be checked at the boundary of every layer until it is finally placed safe and secure into the database. Often if your application is designed well, you should be able to re-use validation logic between layers. This ensures that if the user finds a way to enter data at a lowere level, the data is still constrained.

In summary, Constraint is your first (and in my opinion best) line of defence against potentially malicious users. Next I will discuss the rejection phase.

Technorati tags: Security, XSS, SQL Server Injection, Constrain

Virgin Blue - Sexist Policy

2007-08-26T18:53:00.001+10:00

I travel virgin blue quite frequently, but today on my way home from a visit to Sydney, I ran foul of one of Virgin Blue's policies that I find to be overtly sexist.

I had boarded the plane, and was sitting in the very last row, in the isle seat. I was thinking I must have been extremely lucky when it appeared as though there was going to be no-one else sitting in my row. It meant I could get out my laptop and not have to worry about bumping elbows with the person next to me as I wrote a blog post I am currently working on. Just as they were closing the doors, one of the stewards came down the back with a young boy who looked about 8 years of age, and sat him in the window seat. Still good, the middle seat was still vacant, so I would still not be bumping elbows with anyone. It appeared as though the woman who was supposed to be sitting in the middle seat had not shown up. It was at this point the stewards asked the woman sitting in front of me to change to the middle seat between me and the child. When I looked quizically at the stewardess she revealed that "it is virgin blue policy to ensure that a male is not sitting next to an unaccompanied minor". I couldn't believe it!

I was not given any reason for the policy, I can only assume that with all the media hype around child abuses that Virgin Blue management feel that it is safest to treat all men as potential child molesters, and the obvious assumption, based on a 1920's understanding of gender, is that it is better to have a female sitting next to a child, because obviously a woman would never abuse a child.

I expressed my indignation to the stewards, not that it really bothered me who I sat next to on my hour and a half flight from Sydney to Melbourne, and told them I thought it was sexist. One of the male stewards agreed with me and encouraged me to file a complaint form which I did.

When all is said and done, it may seem like a silly little thing to complain about, and not something that I should get too upset about, but what bothers me is what it says about the kind of society we are becomming. What this kind of policy does is to make males feel uncomfortable around children. We are already at a point where the first thought when an adult male hugs or shows any kind of affection towards a child in public is one of suspicion. I fear that attitudes like this will feed into the already stark gender imbalance in our education system, and into other social activities involving children meaning that a generation of children will grow up not quite knowing how to have healthy relationships with adult males.

Defense in Depth at Tech Ed

2007-08-18T11:35:00.001+10:00

Last week I attended Tech Ed Australia, and as usual at Tech Ed, I found it impossible to stick to one track, instead picking and choosing interesting bits and pieces from almost all of the tracks. One thing that I am starting to get realliy interested in is Security, particularly code security. One thing that I don't think a lot of developers a really aware of is that you can have the best infrastructure security money can buy, you can have fire walls, and DMZ's set up so tight that no un-authorized traffic can possibly get through, but it just takes one bad line of code to blow a whole so wide in your defenses that it can render these measures meaningless.

The buzz these days is on Defense in Depth, which roughly translated means having multiple layers of checks and safe guards so that if a bad line of code up stream lets an attack through, there is a high probability that code further down will pick it up and dispense with it appropriately. I attended a number of security seminars at Tech Ed, and I'd like to expound on a concept that I have have been using in my current role, and it was really good to hear it codified into three basic development pratices at Tech Ed.

The three pillars of this approach to code security are Constrain, Reject and Sanitize, and I want to explain in this series of articles how these concepts fit in with day to day development, and how they can form part of a defense in depth approach to your products.

I'll start in this article by explaining what the three trems mean, I'll then go on and spend an entire article on each of these three concepts and explain how these can fit into the software development life cycle.

Constrain

At the root of almost all code level attacks is user input. SQL Injection exploits the single quote (') special character in the SQL language, Cross Site Scripting (XSS) exploits the web browsers propensity to want to render HTML tags. Generally speaking the developers of the system never intend the input to be in this format in the first place. The constraining of data should be done at the point where the user is entering the data into the system. This is commonly referred to as "white list" checking.

Reject

There are certain known common attack vectors that should be rejected out right. If you are accepting input from a web site comment field, and the user types in "<scrpt>...</script>" chances are you are not really going to want that users comment, and the safest thing to do is to reject it, and tell the user that the input was not acceptable. The rejection of suspect data can be done at multiple phases such as code layer boundaries, or even as an aspect of the way your system works. This is commonly referred to as "black list" checking.

Sanitize

Sanitizing the user input is what I like to call the last line of defense. This is usually done just before user input is presented to the attack target (ie SQL Server in the case of SQL Injection, or the browser in the case of XSS). Sanitizing information is usually done by escaping potentially malicious data before presenting it. As an example, in the case of XSS where the browser is the attack target, a string such as <script>...</script> is rendered harmless if it is HTMLEncoded to <script>...</script>.

Putting them all together

By now you may be able to see the idea of "defense in depth" starting to form. In the first place, we constrain, only allowing data that we expect to be entered. Secondly we reject any known attack vectors so even if we can't constrain every field on in our application (or a lax developer forgets to constrain), we can at least protected ourselves against known attack vectors, and finally if constraining and rejecting fail to pick up a potential attack vector, then sanitizing the output will render the attempted attack harmless.

Readify Developer Network Launched

2007-08-06T16:01:00.000+10:00

An ambitious project by Readify is about to start up. the Readify Developer Network is a way of Readify staff being given the opportunity to learn from and present in front of their peers, and it is open to anyone who is interested.

I'll be speaking on the Ajax Control Toolkit in Melbourne on the 1st of November.

Developing on Vista - Part 3

2007-07-14T16:25:00.001+10:00

Why Move to Vista?

My initial shock at how few people are actively developing on Windows Vista has been tempered by the realisation that I am involved in a very different culture here at Readify, and also the knowlege that Windows Vista is not as revolutionary as it should have been.

The Readify Culture

At Readify we market ourselves as being "Technology Readiness Experts", and as such, we are always playing with the absolute latest technologies that are made available to us, often way before they are even released commercially, and sometimes even before they are stable enough to be used in production environments. This creates, out of necessity, a really vibrant culture of communication. We have a great technical mailing list where we are all avidly asking and answering technical questions, and discussing the ins and outs of the latest software we're grappling with. As much as I am excited to try out the latest things, I am also fairly risk averse. Being a part of the dynamic culture at Readify has given me the confidence to go out on a limb and install Vista, knowing that others have gone before, and that help is only ever an email away. I sometimes wonder, if I wasn't working for Readify whether or not I'd still be developing on Windows XP.

The act of installing Vista has also meant that my own knowlege and experience has increased exponentially, because I am the sort of person who likes to attempt to find the solutions to his own problems before sending out an SOS distress signal. This is why I felt confident standing up in front of a group of 50 or so of my peers and telling them that they can develop on Windows Vista.

Vista's Missed Opportunities

As with most commercial products, there are real world pressures on the development cycle that tend to drive product development a lot stronger than any utopian, computer theory ideal about how things should be, and Vista is by no means an exception.

For me there are two major features that were dropped from Vista because of commercial realities that would have made the imperative to upgrade even stronger. Firstly the fact that Vista was originally supposed to be a fully managed operating system, and secondly the new file system WinFS. I guess it's no use crying over spilt milk, but I do remember feeling a sense of dissapointment as these features were dropped from Vista.

Also in their haste to get Vista out the door, Microsoft released a product that in, my opinion, was not as stable as it should have been. It is only now with many updates installed that I feel Vista is finally starting to behave to my satisfaction.

So Why Bother?

I guess so far I really haven't been making that good an argument for people to upgrade to Windows Vista, and even less of an argument for people to use it as their primary development environment, and to be honest, we've had some consultants at Readify who after trying Vista have decide to go back to developinig on Windows XP. I'm no Microsoft Evangelist, and I wouldn't recommend that everyone use Vista just for the sake of having the latest and greatest, but I think that there are some very good reasons to make the switch.

UI Enhancements

There has been a lot of work done on the way users interact with the desktop environment and how windows are displayed to the user. Some of these features are merely eye candy, other features won't even work unless you have a decent graphics card, but if you do have the hardware, then the way in which you interact with the desktop will be enhanced, and you'll find yourself more productive for it.

There are some really simple ideas such as being able to click on the representation of your application after you press ALT+TAB which I always thought was missing from XP, and when I am forced to use XP at a clients site or a friends machine, I sorely miss.

Integrated Search

Desktop searching is nothing new to windows, but in Vista, Microsoft have made some significant improvements to the performance of the searc, and they've integrated it the OS. Probably my favourite feature of Vista is the fact that search is available from the start menu.

When you click on start (or hit the start menu key), the start menu opens, and places the focus in the search text box. You can then start typing, and windows dynamically finds the best match in your Programs, Files and communications.

As you can see from the image above, you no longer have to remember where The Microsoft Office installer placed Microsoft Word, because you can just click Start, and type "word". You don't even have to wait for the search results to return. Vista Automatically places the best match at the top, and selects it, so if you hit the "Enter" key immediately after you type "word", Microsoft Word will start to run. This saves so much time looking for programs that you don't use often enough to place on the quick launch menu.

Better Security

As discussed in my previous post, I believe UAC is a step in the right direction, although a lot of people are frustrated with it, if people learn to live with it, and use it correctly, it will benefit them. Given that ideally people should NOT have been running as an Administrator when they're only running user applications anyway, Vista makes it so much easier to run as a normal user, that I personally don't want to switch back to developing on XP as a normal user.

Better System Maintenance

There are a lot of system maintenance enhancements that generally go unnoticed (as all good features should). The fact that by default, a hard disk defragment is scheduled to run once a week, means that your hard disk remains optimized without you waiting perform a defrag. There are many such background optimizations aimed at keeping Vista stable and performing at it's peek.

A Word about Performance

Many people have quoted performance as a major reason not to upgrade (or in some cases to downgrade to XP), and this is a valid concern. It's true that older machines struggle with vista, and that things sometimes feel slower on Vista than on XP. My personal experience as been mixed, it started out OK, but as I installed more software, it did get to a point where it was a bit sluggish. I simply went in and had a look at all the services that were running and switched off any that I didn't need, and my laptop is quite responsive now. Also there are some really cool enhancements in Vista specifically aimed at performance. For instance, Ready Boost enables you to plug in a high speed USB 2.0 Flash memory device, to act as another level of caching before data is required from the disk.

Leading Edge

At Readify we like to be on the "Bleeding edge" of technology, trying out things before they are released to the masses. Not everyone is this keen to try out pre-release software, but whether you like Windows Vista or not, Vista has launched and it is here to stay. Yes there are some problems with it, but there will be improvements as time goes on, and hopefully some of the things we find frustrating now will be resolved in service packs. As IT Professionals, people will look to us for guidance, and we need to be able to direct them. I would strongly suggest that if you are thinking of upgrading your development PC that this is the perfect opportunity to take on Vista.

Technorati tags: Vista

Developing on Vista - Part 2

2007-06-22T13:01:00.001+10:00

To UAC or not to UAC?

When I presented on this at the Victoria .Net SIG meeting a week ago, I wasn't quite sure of the general understanding amongst the group about User Account Control (UAC), so I asked the question, "Who here knows what UAC is?", and about 5 (out of 50) people put up there hands. I had intended on launching straight into my argument, but with the vast majority of people not knowing what UAC was, I had to explain a bit about UAC first, so I'll do the same here.

If you are already familiar with UAC, feel free to skip over this explanation and go on to Developing with UAC.

What is UAC?

You may have seen the very funny Mac Vs PC Advertisement that makes fun of UAC, although it is a bit of an exaggeration. Mal-ware has been a serious problem for Windows (as Apple are not backward in pointing out), and the reasons that mal-ware is such a problem stem from the way in which the vast majority of Windows PCs are used. By default, when you create a new user on a new Windows based machine, they are set up as an Administrator (ie have unfettered access to any part of the system, files, registry, hardware etc...). This means that unlike linux or unix based operating systems, you do not need to log in as a user with higher permissions to install software or make system level changes etc..., which significantly lowered the barrier to the general population being able to use PCs, and is part of the reason for the success of Windows as an OS in the home user market. It has also meant a lower barrier of entry to mal-ware which is why we have such a problem. In a unix or linux environment, if a user accidently runs a virus, or if a piece of mal-ware infiltrates the system through a user application, the most that it can do is infect files that the user has control over, which means that it won't infect any system files because a unix/linux user would never dream of running as an administrator when performing normal tasks

It's not like Windows XP forces you to run as an Administrator, in fact in many corporate environments that have a dedicated IT department controlling their network, users are configured to run as normal users on their own machines.Tasks like installing software and configuring networks and drivers are generally handled by the IT department using various remote administration technologies. This makes for a much more secure corporate environment, and means that users can't install malicious software accidentally.

Ideally everyone should be running as a normal (non-administrative) user when doing normal day to day activities, even if the computer is their own personal home PC. The reason people don't do this is 2 fold. Firstly because it is the default when installing a new operating system, and secondly, a lot of applications have been written for windows that for no good reason require Administrative access to system resources. So UAC is Microsoft's first attempt at fixing this problem.

In Vista by default when you create an Administrative user, Vista will generate 2 separate user tokens, one that has Administrative access to the system, and another that has normal user (restricted) access to the system. When the user is logged into Vista, by default the system uses the token associated with the normal user, but if an application attempts to do something that requires elevated privileges, Windows Vista will warn the user with a dialog asking if they started the program, and if it's OK to use the Administrator token to perform the operation. This means that if malicious software tries to infiltrate the system without being detected by the user, as soon as it attempts to do anything that requires administrative access, it's cover will be blown, and the user alerted to what's going on.

As with many things in Windows, you can actually turn UAC off completely, however, this just puts the user back into the position you were running Windows XP with Administrative privileges. A better approach is if an application requires administrative permissions (and really the only types of applications that should are system utilities that are performing low level administrative style tasks, not your general run of the mill user application), then you should use the "Run As Administrator" option to start the application. This means that you give that application the permission to run using your administrative token, while everything else uses your normal user token, protecting you from malicious software.

Now I'll be the first to admit that UAC isn't perfect, and especially as a developer, I do find myself getting a lot of UAC warnings as I perform certain tasks, but I personally think it's a step in the right direction. I also believe that for most normal users, these UAC warnings will happen so infrequently that they shouldn't be a problem.

Developing with UAC

Visual Studio 2005

When Vista first came out, and the early adopters in the development community first started using it as their primary development platform, there were many issues. One thing that did happen when you attempted to run Visual Studio as a normal user is that it would display a warning saying "You should run Visual Studio as an Administrator". In fact this was the official line from Microsoft, and as far as I'm aware still is. I also believe that this is WRONG! Service Pack 1 and the Updates for Windows Vista have fixed a lot of problems, and it is now possible to run Visual Studio and perform MOST of the tasks you do as a developer in normal user mode. The majority of things work just as they did on Windows XP. Developing Console Applications, Winform Applications, WPF Applications, and many more types of applications DO NOT require you to be running Visual Studio as an Administrator. However, there are some scenarios where you may still need to run Visual Studio as an Administrative user.

Most of the problems revolve around trying to debug into processes that you don't own. For instance if you are developing an ASP.Net application and you are using IIS7 instead of the the Visual Studio Development Webserver (Cassini), then debugging requires you to attach to the w3wp process. Not surprisingly, this requires you to have Administrative Privileges, and currently there is no way from within Visual Studio to elevate those permissions on the fly, so if you are intending on doing this, you'll need to start Visual Studio as an Administrator. Alternatively, you could use the Visual Studio Web Development Server (Cassini) to host your application while in development. Cassini quite happily allows you to attach and debug into it as it is started by the same user that started Visual Studio. It also has the advantage of allowing the "edit and continue" feature of Visual Studio debugging (something IIS7 or IIS6 can't do), and it is generally a little faster. Cassini will behave identically to IIS7 in the vast majority of scenarios, however there are some deficiencies with Cassini. It can't use the HTTPS protocol, in this case I usually find myself adding a configuration setting to determine if I am in development or production, and switching to use HTTP when in development. It also can't emulate subdomain scenarios properly. So if you need to test Single Sign On scenarios across subdomains, then you will need to use IIS7. There are also some other subtle differences in the way Cassini processes the ASP.net pipeline to the way IIS processes it. I am a firm believer in testing your application as close to your final production environment as is practicable, so I do recommend that final stage development testing is done against IIS, and not cassini, but you should only need to debug using IIS7 if you find a problem that only occurs when running the application under IIS7, and does not exhibit when running under Cassini.

Another thing that I have found to be problematic is installing assemblies into the GAC. Installing assemblies into the GAC is by it's very nature an administrative task, and requires elevated privileges. I have seen many projects that, as part of a post build step automatically deploy the newly created version of a shared assembly to the GAC. Obviously if you are running as a normal user, this post build step will fail, thus causing the build to "fail". Now personally I would question the need to install libraries to the GAC during development, and would encourage development teams to look at ways of privately deploying shared libraries during the development phase instead of deploying to the GAC. Adding a probing element, or a codebase element to the applications configuration file will allow you to share common assemblies amongst applications during development. However, if you insist on installing libraries to the GAC in a post build step, then you will have to run Visual Studio as an Administrator for that project to build.

There are some other minor glitches that are left over as a result of Visual Studio 2005 not quite being vista ready, (see Issues running Visual Studio 2005 on Vista as a normal user). Even running Visual Studio 2005 with elevated permissions there are still some minor problems on vista, (see Issues running Visual Studio 2005 on Vista With Elevated Privileges), but these issues occur infrequently enough, or there are adequate workarounds for them not to be a problem (at least in my experience).

SQL Server 2005

When Vista first hit the streets, SQL Server 2005 Service Pack 1 had already been out for some time, but unfortunately there were still lots of incompatibilities with Windows Vista. Most of us were forced to run SQL Server Management studio as an Administrator to tackle some of these issues. Service Pack 2 fixes a lot of these problems, and certainly after running the user provisioning tool, there is no longer any need to run SQL Server Management Studio as an Administrator anymore. There are still some issues mainly around Microsoft Visual Studio for Applications (VSA) used in SSIS script tasks, but beyond this, there are few significant problems.

See Windows Vista Considerations in the Service Pack 2 Readme document.

Why not just turn UAC Off?

Many developers argue that they are experienced computer users and as such feel somehow immune to mal-ware, and anyway, they run anti-virus software, so they are protected against malicious software attacks. They also argue that they constantly need to perform tasks that require administrative privileges, and therefore feel that turning UAC off all together will save them heaps of time, and put an end to all the UAC warnings.

Although dubious, let's assume for a moment that the first argument holds some water. I hope by now I will have been able to convince you that you don't need to run Visual Studio as an Administrator all that often. I would also argue that the effort of Clicking "Continue" from time to time on UAC dialogs when you do need to perform Administrative tasks is not going to cost you that much time in the grand scheme of things.

Another argument to keep UAC switched on is put very eloquently by Ian Griffiths, in his post UAC: Don't be part of the problem. Ian argues (as I did in my explanation of UAC) that part of the reason people feel they have to run as an administrator on Windows is because there are so many end user applications that require Administrative privileges for no good reason. He then goes on to argue that the reason that so many applications require elevated privileges is that they were written by developers who were running as Administrators on their own machines at the time of development, and were completely unaware that the code they were writing would not run without elevated privileges. he quite rightly says that this sort of behaviour in the linux/unix/OS X development community would be frowned upon, and the resultant applications would just not get used. Please DO NOT turn off UAC before reading Ian's article.

I would also argue that always running Visual Studio as an Administrator is just as bad (at least from the point of view of Ian Grifiths' argument) as turning UAC off altogether, because when you debug your applications, they will always be launched with elevated privileges, and you will never see potential normal user problems.

I come from a position of having developed as a normal user on Windows XP, and it is soooo much easier to develop with UAC on Vista than as a normal user on Windows XP. It is also much more secure. If you were developing as a normal user on Windows XP and needed to run something with elevated privileges, you would have to use the "Run As" command which would require you to type in your administrator password every time. The more times you are forced to type in a password the more chance there is of over the shoulder credentials theft. UAC simply requires you to validate that it was you who invoked the task that required the elevation of privileges.

Tips to help you stay on the wagon

Here are a few tips to help you run on Windows Vista and navigate the potential problems of UAC.

Add Visual Studio 2005 to the quick launch menu

This not only makes it quick and easy to find, but on the rare occasions when you do need to run Visual Studio as an Administrative user, you can simply right click the icon, and select "Run As Administrator"

If you find yourself needing to perform lots of administrative tasks, simply run a command prompt with elevated privileges and keep it open as you develop. Anything you run from this command prompt will automatically have administrator privileges, and you won't get asked to confirm the elevation of privileges.

I am sure that there are a lot of people out there developing on Vista who have come up with some really good stratergies for developing on Vista, and I'd love it if you could leave any other tips/suggestions as comments on this post.

Conclusion

If there is one thing I hope you take from this post it's you don't need to turn UAC off to develop on Vista. Also that you don't always need to run Visual Studio 2005 as an Administrator all of the time.

Links

Developing on Vista - Part 1 : Installing the toolset

Technorati tags: Vista, Visual Studio, SQL Server, UAC

Developing on Vista - Part 1

2007-06-20T09:15:00.001+10:00

As promised, I am putting together a series of articles around setting up your development environment on Windows Vista. I have been using Visual Studio 2005 and SQL Server 2005 (amongst other development tools) on Windows Vista since November 2006, and I must admit that the experience hasn't been all plain sailing.

However, with the release of service packs for both Visual Studio 2005, and SQL Server 2005 that address some of the problems, the situation now is much better than when I started. There has also been a lot of mis-information disseminated about how Visual Studio 2005 and SQL Server 2005 interact with UAC leading to some confusion about how we should run them.

In Part 1 Installing the toolset I hope to give some guidance around getting your development environment installed and running. In Part 2 To UAC or not To UAC? I'll explain the issues around User Account Control (UAC) in some detail, and hopefully convince you that you don't need to run Visual Studio 2005 as an Administrator all the time. In Part 3 I'll attempt to answer the question Why Move to Vista, and in Part 4 Tidying up I'll discuss what's not supported, give some general tips and tricks for developing on Windows Vista, and link to some further reading that might be useful.

Installing the toolset

As developers we all have various tools and utilities we just can't live without. I am a .Net developer, and as such I wish to limit this discussion to the Visual Studio 2005 and SQL Server 2005 toolset. I will talk about a few of the other tools that I have had dealings with, but it will by no means be a complete list. I encourage anyone who has experience (good or bad) with a particular tool on Windows Vista to leave a comment on this post.

Choose a Vista

The first step when taking the plunge into Windows Vista, is to decide on what version of Vista is appropriate SKU of Windows Vista. The good news is that Visual Studio and SQL Server are supported on all SKU of Vista. There are however some issues when running Visual Studio 2005 on versions of Vista that don't support Active Directory, so my general advice is to choose one of the following versions

Order of Installation

Since I started using Visual Studio 2005, and SQL Server 2005 back in January 2005 (yes that's right almost a year before they RTM'ed), I have installed the toolset many times, and had some success as well as some pain. I have developed the following order of installation based on this experience, and also on my experience since starting using Vista (November 2006).

Set up IIS 7
Install SQL Server 2005 Service Pack 2
Install Visual Studio 2005 Service Pack 1 With Vista Updates
Install any other tools that you can't live without.

I will go through these step by step.

Set Up IIS7

By default IIS7 is not installed when you first set up Windows Vista, this is part of Microsoft's attempts to reduce the potential attack surface of the base OS, and make it a more secure environment for everyday users. However, if you're intending on doing any ASP.Net development, or want to run SQL Server Reporting Services, you'll need to install it and configure it appropriately.

To Install IIS7, Go to Start->Control Panel->Programs and Features, and select "Turn Windows Features on or off". In the "Windows Features" dialog, select "Internet Information Services", and ensure that you enable the following features as a minimum.

IIS 6 Management Compatibility

Without this, you can sometimes get some cryptic errors when performing certain tasks about not having Microsoft Front Page Extensions installed"

.Net Extensibility
ASP.Net support.

Figure 1 shows how this might look.

Figure 1

Install SQL Server 2005

Obviously you need to decide on what SKU of SQL Server you want to use. For developers this usually comes down to a choice between SQL Server 2005 Developer Edition and SQL Server 2005 Express Edition. I personally prefer to run SQL Server Developer edition on my primary developer environment because I often use features that are not available in the Express version. This does not mean that you can't install SQL Server Express as well if you need to develop against it, but I would suggest installing the developer edition first. So the sequence would be...

Install SQL Server 2005 Developer Edition
Install SQL Server 2005 Service Pack 2 (282.4 MB - 392.4 MB)

Perform User Provisioning*

Install SQL Server 2005 Express SP2 (optional)

User Provisioning allows you to add Vista users to the sysadmin fixed server role, make sure you provision the user you develop under.

* Be careful when performing the user provisioning immediately after the Service Pack install. Because the service pack requires you to stop ALL SQL Server Services, and doesn't re-start them before asking you to perform User Provisioning, the user provisioning will fail unless you explicitly go and re-start the SQL Server services. Failing this, you can always run the User Provisioning tool from your SQL Server installation at a later point.

C:\Program Files\Microsoft SQL Server\90\Shared\SqlProv.exe

Install Visual Studio 2005

Again, you'll need to choose which SKU of Visual Studio you wish to run, but once you have made the choice, the installation procedure is as follows.

Install your flavour of Visual Studio 2005
Install Visual Studio Team Explorer (if you are intending on connecting to TFS).
Install Visual Studio 2005 Service Pack 1 (431.7 MB)
Install Visual Studio 2005 Service Pack 1 Update for Windows Vista (28.9 MB)

If you wish to start writing .Net 3.0 applications, then you'll also need to install the following

NB the extensions for WPF and WCF is a CTP release, and as with most CTP software, there are some problems with it (running either on XP or Vista), and as far as I'm aware, this is the state it is remaining in until Orcas is released (I guess MS need to have something to encourage people to upgrade to Orcas when it comes out).

Install any other Tools you can't live without

At this point, you can then install any other tools/utilities/plug-ins that you normally use. Just check with the manufacturers around each tools compatibility with Vista, I will discuss in a later post some tools that have issues, but my experience on the whole is that most tools work fine on Vista.

Conclusion

This will get you to a point where you are ready to start developing on Vista. Most things should work just as they did on Windows XP, however, keep in mind that these tools were specifically written for Windows XP, and there are still some notable exceptions to this rule which I will cover in my next post.

Technorati tags: Vista, Visual Studio, SQL Server

.Net SIG Presentation Wrap up

2007-06-19T11:41:00.001+10:00

Well, I must say I got a bit of a shock when I asked "How many people are using Vista as their primary development environment, and 4 people out of 50 put up their hands. Maybe it's just the Readify culture, but most of us at Readify are using Vista as our primary development environment, (some at Readify have been using it since before RTM), and I had expected that a significant proportion of Developers would by now be using Vista as their Development platform.

This surprise sparked a question. "Why is it that so few Developers are running Vista as their development platform?" I'm not suggesting that Vista will suit everyones needs, or that people should upgrade just for the sake of upgrading, but developers are generally NOT techno-phobes, and are usually fairly quick to adopt new technologies that increase their productivity, or even just for the novelty value.

I guess there are numerous reasons why this might be the case, but one that occurs to me is that the information required to set up your development environment on Windows Vista seems to be spread around various places on the web making it a significant research task to find all the details. There has also been some mis-information that has lead to confusion. This I feel I am in a position to rectify. As a resut I have decided to do a series of posts aimed at accumulating the best practices around setting up your development environment on Windows Vista.

As I promised in my presentation, here is a link to the slides from my presentation, however, I will over the next few days put together a series of posts that cover roughly the same concepts as in my presentation, but go a little bit more in-depth.

Latent bug syndrome

2007-06-08T13:59:00.001+10:00

Today I have fallen foul of what I call the "Latent Bug Syndrome". This will hopefully help people understand just how complex software engineering is, and why bugs occur in software despite the best efforts of all involved. Let me explain.

Let's say you have a system like the following.

where System A represents a fairly complex, fully featured, fully tested product that exposes cerrtain functionality to the end users. Now System A can be operating 100% perfectly (or at least for the overwhelming majority of scenarios that are encompassed by the exposed feature set), however there still may be a "Latent Bug" hidden deep within the underlying logic of this system that is never exposed to the end user because the exposed feature set never actually allows the system to get into the state that would cause this bug to exhibit. As a really contrived example, let's take the classic divide by zero problem

public stat double DoCalc(double val)

{

return 42 / val;

}

This function operates as expected under all conditions except where val = 0, in which case it throws an exception. If the exposed feature set only alows the user to select numbers from a drop down box that contains the numbers 1, 2, and 3, then every possible test that a tester can do will NEVER produce an error. Keep in mind this is possibly the simplest example one can find, in real life, the scenarios are far more convoluted. The wrong assumption from this is that System A has no bugs.

Now lets suppose that System A is augmented with System B as shown below.

Now System B has the potenial of producing conditions in System A that not only were never tested, but were never even thought of by the original developers and testers of System A, and in my contrived example, a user input into system B may very well cause the value of the val parameter to be passed 0. The thing is that the developers (and testers) of System B are usually different to the developers (and testers) of System A, and often don't have access to the source code, or even if they do, don't have time to follow every code path through the interactions with System A accounting for every possible state that System A could possibly be in to find such a bug.

If the testers of System A are doing a good job, they may actually pick up the bug, then of course the blame game starts. Who is responsible for this bug, who's going to fix it why wasn't this bug picked up before etc....

So this happened to me today. Fortunately the bug never made it to a production system (and no it wasn't the divide by zero bug suggested above, it was infinitely more intricate), but it did make me aware that I had made some assumptions about System A that turned out to be less than 100% accurate.

So how can one guard against The Latent Bug Syndrome.

System A

Thorough unit tests of all public (and some times even private) methods exposed by system A will catch a vast number of these bugs.
Better documentation of publicly exposed methods to help developers integrating with your system understand what assumptions you are making about the input to a method, and what state you are expecting the system to be in.
Test Driven Development could actually go a long way to irradicating this all together because in TDD you only write code to satisfy the tests nothing less, nothing more, so you don't get the situation where there is code that under ALL conditions tested is never executed. However, I have yet to see a company embrace TDD to that extent as the sheer time involved is usually far too much to justify it commercially.

System B

Developers and testers of system B need to have a good understanding of how system A works, what it was intended to do, and how they are changing the way in which System A is being called. Understanding this may help you know where problem are likely to occur.
Know what your assumptions about System A are... and constantly question them.
Again unit tests are your friend.
wrapping your interaction with System A is also a good way of being able to respond to changes in System A if they occur.
Thorough end user testing (there is NO substitute).

VS2005 Project Templates Failing

2007-06-06T21:44:00.001+10:00

I came across a problem recently where I went to create a standard Console Application project in Visual Studio 2005 and to my alarm I got a message saying

---------------------------
Microsoft Visual Studio
---------------------------
Could not find file 'C:\Program Files\Microsoft Visual Studio 8\Common7\IDE\ProjectTemplatesCache\CSharp\Windows\1033\ConsoleApplication.zip\ConsooleApplication.csproj'

---------------------------
OK
---------------------------

This took me back a bit, as creating a Console application is one of the most basic tasks you would ever want to do in Visual Studio 2005. my next thought (after trying exactly the same thing 5 or 6 times) was to run visual studio with elevated priviledges, but alas this still didn't fix the problem. I checked and sure enough I could still create a Windows application no problems (with or without elevating my priviledges), so I just put it down to some kind of vista weirdness and continued on (using Winforms apps instead of console apps whenever I wanted to create a quick test). Today, however, I discovered that I could no longer create Web Service Applications as they crashed with a very similar error, and with a presentation on setting up your development environment on Windows Vista looming, I decided that I had better get to the bottom of the problem.... and so I did.

It turned out to be the Guidance Automation Toolkit and Guidance Automation Extensions packages I recently installed that for some reason had done some damage to the Project Template System in Visual Studio. now I guess fair is fair, both of these packages are in beta, and there are some known issues around running them on Vista (unfortunately I was not aware of these issues at install time). One of the big issues with the Guidance Automation Extensions is around uninstalling it. You get so far, and then a dialog pops up, no title, no message, just an OK button, and it stops uninstalling. fortunately I found a blog post by one Greg Duncan on how to uninstall the Guidance Automation Extensions.

I can now happily create console and web service applications again, but more importantly stand up in front of the Victoria .Net SIG next tuesday and talk confidently about running Visual Studio 2005 on Vista.

Speaking at Victoria .Net SIG

2007-06-05T14:00:00.001+10:00

Just a quick note to say that I'll be speaking at the next Victoria.Net SIG meeting on the 12th of June at Microsoft building, Level 5, 4 Freshwater Place, Southbank. Turn up for Pizza and soft drink at 5:30 pm.

The topic of my discussion will be setting up your development environment on Windows Vista. Hope to see you there.

IT a series of stops and starts

2007-05-03T00:02:00.001+10:00

Is it just me, or does anyone else in the industry feel that working in the IT industry is a series of stops and starts. What I mean by this is that often when i start on something that I think is going to (or at least should be) faily straight forward, quite often, very early on in the peice I come across a problem that consumes way more time than I'd expected. For example, today, I was asked to fix a bug that was occuring on a project that I was involved with no less than a few weeks ago. I know the codebase had changed significantly, as a merge with the main branch had been done. What I didn't expect was that I'd spend the first 3 and a half hours tryng to get my machine to a point where I could compile and then run the project. Now part of the problem was my grappling with the back up and restore features of SQL Server, and one could argue that I potentially should understand that better (and now do), but the rest of the time was grappling with issues that we have been having with bugs inTFS that cause issues when trying to Get the latest versin of all files in large projects. Then there are aother times like right now where I am trying to do a few things on my PC, and even though I have a fairly new DELL D820 laptop with 2GB of RAM, and 4GB of ready boost, the hard drive is churning continuously (not sure why), and it takes up to 10 seconds to switch between tasks. This may have something to do with the fact that about 20 minutes ago I had to actually reset my computer after it crashed while trying to use Explorer. I am fast approaching "a sad realisation" that although windows Vista may be more secure, it is not in any way speedy, or stable. I really think Microsoft rushed it out the door, and that the end user is paying the price. It's times like these when I start to think about work in other sectors, hmmmm.... maybe I'll start up a Tango dance studio.

Tango performance

2007-04-22T11:21:00.001+10:00

Just to show how behind the times I am, here is my first ever appearance on You Tube. This is a Tango performance Niki and I did last weekend at our friend Yeow's birthday party. Niki and I have been dancing Argentinean Tango for 6 years now, and it is becoming an all consuming passion.

Technorati tags: Tango

It's had its fair chance, time to switch back

2007-04-18T11:06:00.001+10:00

That's it I've had it with IE7, I used to use firefox for the vast amount of my work until IE7 came out, and in the spirit of giving IE7 a fair trial, I decided to use it as my default browser. I started this back when I was still running Windows XP, and I was then using IE7 Beta 2, and the odd crash was acceptable for a beta product. I liked the tabbed browsing, and thought theat they actually did some things slightly better than firefox. I was impressed with the efforts made in standards support, and for this reason, I had persisted with it, even though the rendering engine is significantly slower (up to 5 times slower in some cases). When I started running Visat in november last year, I noticed that from time to time IE would crash (even though it was now out of beta), but I thought "patience... Microsoft are fairly good at stabilizing their products.... eventually, and the release of Vista was a rather rushed one, just so they could get it out the door before the end of the year, so I can expect that there may be some bugs that they find and fix along the way". Well, it's now mid April, and Internet Explorer is still crashing quite frequently, and today as I was browsing some Micosoft web sites, it crashed again. This is the final straw, I think they've had long enough to fix these problems. I've just switched back to firefox as my default browser.

Blog posts as an indication of my state of mind

2007-04-16T22:20:00.000+10:00

When I started blogging way back at the start of 2004 (see http://scottbaldwin.blogspot.com/2004_03_01_archive.html, (I'd like to say before blogging became popular, but that would definitely be a lie) I was in a very stressful job that I didn't really enjoy, it was taking up far too much of my time, and was making me feel quite frustrated with the IT industry to the point where I was almost ready to leave the field for good. So even though I found out about blogging through my work in the IT field, the first blog I started was nothing to do with IT at all, in fact musings of a morbid mind was where I intended to post all of my political rantings and ravings, my general thoughts on topics (generally outside of IT), family and travel photos etc... , and just a general writing exersize for me. I started my geek blod Daily Dribblings of a Demented Developer shortly after (May 2004), but it was always the poor cousin of my musings (btw no prizes for guessig that alliteration is one of my favourite literary devices). I started work for Readify at the start of 2005, and since then, my writings on my Geek blog have steadily increased, and my writings on my musings blog have decreased, to the point where I noticed today that my geek blog has almost caught up, and what's more surprising the blog articles I've got in the pipeline are almost ALL for my Geek blog.

Its amazing the amount of difference working for a good company can make, there are still aspects of the IT industry I'm not enraptured about, and I still get stressed from time to time when things don't work the way I expect them to, but I have come to realise that there is a lot I do enjoy about IT.

Fixing the WCF - SSIS Web Service Task problem the easy way.

2007-03-25T15:47:00.001+10:00

Literally minutes after publishing my post on calling a WCF Web Service from the SSIS Web Service task, I checked back with the forum post that I'd mentioned in that blogpost http://forums.microsoft.com/TechNet/ShowPost.aspx?PostID=1023821&SiteID=17 only to discover that Uwe Heinkel had given me the peice of information I was missing in the first place. So the crux is I don't have to hand craft the dtsx xml file to get it to play ball.

I simply remove the following line from the wsdl file

<xsd:import schemaLocation="http://localhost:50344/MyWebService/Service.svc?xsd=xsd2" namespace="http://schemas.datacontract.org/2004/07/"/>

and it all just works as expected.

damn.... I was having soooo much fun wading around in the dtsx xml file.

Calling a WCF Web Service from the SSIS Web Service Task

2007-03-25T15:10:00.001+10:00

I have recently been doing some work using SSIS, and this is really the first time I've used for anything serious (ie beyond doing the tutorials that come with SQL Server 2005). I have also recently been learning WCF and as we needed to create some Web Services to deliver the information to SSIS, I decided that this would be the perfect opportunity to put my newly acquired WCF skills to good use. Now I always like to start things out very simply, which in this case meant prototyping the SSIS Web Service Task with the most basic of WCF Web Services I could possibly create, thinking once I get this working, I can then move on to the actual task that I want to achieve with confidence... unfortunately this is where my whole plan started to unravel. For those following at home, here are the steps I took.

1. Create a new WCF Web Service in Visual Studio 2005 by selecting File->New->Web Site, then choosing a WCF Web Service. (Note .Net 3.0 and related Visual Studio extensions are required for this. Also choosing a File based or IIS based web service shouldn't matter).

2. Open the web.config file change the binding to basicHttpBinding (although the same end result is still achieved with the default wsHttpBinding).

3. Expose the metadata over httpget by adding the following line to the service behavior in the config fil

4. run the service.

5. Create a new SSIS project in Visual Studio by selecting File->New->Project, then choosing Business Intelligence->Integration Services from the menu.

6. Add an http connection that points to the wsdl file for your service, and a Web Service Task to the Control Flow that uses the http connection.

7. Edit the Web Service Task, and download the wsdl exposed by your service, and save it locally to a file by using the OverwriteWSDLFile = true on the General page.

8. Select Input, and then select 'MyService' from the drop down.

At this point I recieved an error message.

Item has already been added. Key in Dictionary: ‘DataContract1’ key being added : ‘DataContract1’

It then would NOT populate the Method DropDown, so I was unable to select a Service Method to call, and because you are confined to selecting from the drop down, there is no possible way to make any of the Web Service method calls.

Now I'm the superstitious kind when it comes to writing software, my firm belief is that if you can't get the most basics Web Service working, then you may as well not try to get your complex web service that wants to pass around meaningful data, working, you'll end up in tears.

A google search on the problem yielded the following forum post http://forums.microsoft.com/TechNet/ShowPost.aspx?PostID=1023821&SiteID=17 which was helpful in revealing the underlying problem. The problem is with the generated wsdl and corresponding schemas, not sure exactly what is wrong with them, but for some reason, the UI in Visual Studio doesn't like the way WCF publishes its metadata, in particular the way it published metadata about its DataContracts (complex types). So I thought I'd make the default WebService that WCF generates even simpler by removing any trace of the DataContract. This worked fine, I could call a WCF WebService from within SSIS no problems... with the restriction that I could only pass primitive types to/from the web service. now this might be fine for your purposes, but I required something a little more powerful, so I decided to persist with figuring out the solution.

I tried a number of things with the generated schemas, but I think in the end I'd exhausted my limited understanding of wsdl and xsd, and was no further ahead. Next I remembered a very simple statement by a fellow collegue of mine Andrew Ball who was at the time giving me some SSIS advice. He said "... under the hood an SSIS package (dtsx file) is just a big xml file". Armed with this I began to see the issue as more of a tooling issue, and thought that maybe the issue is purely with the designer. So I cracked open a sample that Andrew had sent me and found where the various Web Service properties were being set. I also cracked open good old reflector and took a look at the way in which Complex Types were handled in the Microsoft.SqlServer.WebServiceTask.dll Which is responsible for the Web Service Task in SSIS. about half an hour later (give or take 10 mins) I had the following XML

<WSTask:MethodInfo WSTask:MethodName="MyOperation2" WSTask:MessageName="">

<WSTask:Documentation>

Basic Hello world service call that uses a DataContract

</WSTask:Documentation>

<WSTask:ParamInfo WSTask:Name="dataContractValue" WSTask:Datatype="DataContract1" WSTask:ParamType="Complex" WSTask:SeqNumber="0">

<WSTask:ComplexValue>

<WSTask:ComplexProperty WSTask:Name="FirstName" WSTask:Datatype="string" WSTask:ParamType="Primitive">

<WSTask:PrimitiveValue>Scott</WSTask:PrimitiveValue>

</WSTask:ComplexProperty WSTask:Name>

<WSTask:ComplexProperty WSTask:Name="LastName" WSTask:Datatype="string" WSTask:ParamType="Primitive">

<WSTask:PrimitiveValue>Baldwin</WSTask:PrimitiveValue>

</WSTask:ComplexProperty WSTask:Name>

</WSTask:ComplexValue>

</WSTask:ParamInfo>

Which when placed inside a web service task element, can be used to successfully call the web service with the DataContract.

So my conclusion is that there is a bug in the SSIS Web Service task UI that doesn't handle populatinf the Method drop down correctly (I'd trust WCF before I'd trust SSIS), but this is merely a bug with the UI, not with the way in which the Web Service task actually goes about calling the web service.

Added after post (literally minutes after): For the easy way to solve this problem see http://sjbdeveloper.blogspot.com/2007/03/fixing-wcf-ssis-web-service-task.html

Flickr becomes alive

2007-03-01T08:51:00.001+11:00

Just found a really neat Plugin for Windows live writer called flickr4writer that allows you to add images from your Flickr photos by simply clicking "Insert->Flickr image...", so to celebrate here is one of my flickr photos.

Molly helps Microsoft with Standards

2007-02-27T19:35:00.001+11:00

This is huge... I've been up against some major deadlines so I haven't been keeping up with all my blog reading, but today I decided that before I go home I was just going to catch up on a few of the more important ones, and so I was reading the IEBlog when I stubled across this article, the basic crux of which is that Molly E. Holzschlag has "signed on with the Internet Explorer team on a contract basis to work on standards and interoperability issues".

I have actually heard Molly speak at the Web Directions South conference last year, and she is a really inspiring speaker, and someone who tirelessly advocates for better adoption of Web Standards. It'll be interesting to follow her progress at The Daily Molly.

Published articles for The Readify Tech news letter

2007-02-03T12:38:00.001+11:00

Just thought I'd mention that I now have a published two part article on the Readify tech news letter on SQL Server Express Edition.

You can view the articles at

SQL Server Express (Part One) - Another reason to move your user applications onto a stable platform

and

SQL Server Express (Part Two) - How User Instances Work

It finally dawned on me why I don't like the name "ASP.Net AJAX"

2007-02-03T12:27:00.001+11:00

Microsoft renamed their AJAX offering from ATLAS to ASP.Net Ajax around November last year. Now I personally never quite understood what was so bad about ATLAS, but I do realize that as a general rule, product code names generally have to change at some point before the release to the main stream, (Longhorn -> Vista, etc...), but I must confess that quite often Microsoft completely screw it up.

I guess there is significant pressure from various departments, notably marketing and branding who want to maintain a consistent theme, and have the morbid fascination with trying to imbibe descriptive meaning into product names, and also from the legal department who are desperately trying to avoid trademark infringements. This is why I didn't blog about the name change when I first learned of it, but I always felt a huge sense of disappointment about the change, but couldn't quite put my finger on why I was so disappointed... until this week.

Beyond the obvious lack of imagination displayed taking the two core technologies (ASP.Net and AJAX), and just combining them (which in itself causes some confusion), I realised something this week when I was attempting to debug an issue with the library. As a developer one of our most important tools is our favourite Internet search engine. Whenever we come across a problem say with a CSS styling that isn't displaying correctly in a browser we just go to our favourite search engine and type in a question, for example "CSS min-width Internet Explorer". In Google the first page of the previous search returns all types of useful hits, including information regarding the fact that min-width is not supported in Internet Explorer 6.0 and a number of ways you can work around this. This is very powerful and allows us to do our jobs a lot quicker.

I discovered when trying to debug a problem with ASP.Net AJAX that this using ASP.Net and AJAX is not only longer to type than ATLAS, but returns all kinds of results that you wouldn't really be interested in. If you just type in ASP.Net AJAX into Google, the first page returns (as you'd expect), the official home page (ajax.asp.net), and a few other relevant topics, but then it starts to get more off-topic, like other component vendors offering AJAX style controls in ASP.Net such as ajaxium, ajaxpro, etc..., this is then further confounded if you need more search terms. The search is helped marginally by quoting "ASP.Net AJAX", but still returns that ajaxium website in the first page of results.

If Microsoft had've done what the trend is in the rest of the industry and actually used some imagination to come up with a short sharp name that was unique in the industry space, it could've made all our lives a little bit easier.

Dell Quickset disabling Internal network card

2007-02-02T10:13:00.001+11:00

I had an interesting network problem today. I arrived at work, plugged my laptop into the power, and then into the network cable, switched it on only to find that I had no network connectivity. After rebooting twice, and discussing it with my colleague sitting next to me who has pretty much an identical laptop, connected to the same router, with no such networking problems, I did a bit of poking around.

I have a latitude D820 running Windows Vista, and the Dell Quickset power utility. One feature of quickset is that it disables the internal network card if you are not using it and you are on battery power, to get a few more minutes of battery life. It turns out that it must be a bit buggy as this turned out to be my problem. It had disabled my network card from the night before, and admittedly I had suspended my session, but then this morning, even after plugging into the power at work, and after rebooting my system twice, my network card still failed to respond. I fixed it by explicitly setting in the quickset UI to "always activate on battery".

IE Developer toolbar rocks

2007-01-16T17:47:00.001+11:00

Just downloaded the latest beta of the IE Developer toolbar, and I must say it seriously rocks....

I've been using previous versions for a while now, and have found it extremely helpful in debugging styling problems, but this version goes to the next level. I haven't had a good chance to put it through it's paces yet, but I'm sure the "Style Trace" functionality will prove indispensable. It'll be interesting to see what will be in the final version of this product, they may even start to challenge firefox as the "Developers preferred browser", although I think they still have a way to go there.

Web Directions North gets the nod of approval from Microsoft Luminaries

2007-01-16T16:30:00.001+11:00

Just read a post on PeterL's blog, about Web Directions North. I went to the Web Directions South conference myself. Here's my wrap up of the Aussie version, although I must admit that my post doesn't do it justice, the event is well worth attending, and will challenge the way you think about the web.

Warning: if you do anything of note during a session, you'll probably see yourself used as an example of the power of web 2.0 as the speaker pulls down the flickr image of you minutes after the event.

tags: wd06

Back and Live

2006-12-22T16:25:00.001+11:00

After much frustration with Windows Live Writer (Beta) and Blogger (Beta) having communications issues, I am now back blogging with Windows Live Writer.

Gotta love trying to get two beta products communicating.

Technorati tags: blogger, Windows Live

As Kermit would say ... "It's not easy being Green"

2006-12-04T22:10:00.000+11:00

A fellow collegue of mine Darren Neimke has posted a rather interesting post on his blog, which has got me thinking. Now I have to admit, I do consider myself to be a "Greenie", however, far from having "jumped on the bandwagon, I have been a member of The Australian Greens now for well over 3 years, and in fact I can trace my pilgramage from the high slopes of comfortable numbness about environmental issues into the radical valley of the shadow of Greenie Activism to a course I took back at uni, almost 11 years ago. A philosophy course aimed at Engineers called Technology and Human Values in which we studied "systems engineering" approaches to the ways technologies are used and abused both here and in developing countries.

Anyway, my main reason for writing this blog article is to suggest to Darren, that I feel has has understated some of the issues slightly. I am ussuming that some of the understatements were intentional and for effect like " .... Or some ice melting in Antarctica." trivialising the disapearance of entire glaciers, however, I the one I really want to concerntrate on is the issue of research and the problems of people reading something and just believing it. From his post

Research is a skill that is being diminished by the one-click world of Google and the new media. People hear something and believe it to be true without even having the faintest clue of how they would ratify such assertions.

I think in this statement are a number of problems all rolled into one.
Firstly, yes we live in a society based on instant gratification. Google is a great tool, that can be used to justify any point of view you want to throw at it.
Secondly, I think it is quite difficult to "ratify" certain facts. The problem comes from the nature of the beast that is science. To really fully understand the latest scientific findings, requires you to pretty much be at the forefront of scientific reseach in that particular area, and this requires not only having a PHD, but being actively involved in hands on research. The problem is that even scientists at this level can't always seem to agree, and the way in which research projects get funding often requires researchers to make bold (sometimes even rediculous) statements to gain the attention of the various institutions that are willing to put up the money to fund this research. How is your average punter expected to be able to know what the truth is even if they have tracked down the information from people considered leaders in their field.

Not wanting to be all doom and gloom, I think that there are solutions emerging to these problems.

If I may use a buzz-word, with the web 2.0 paradigm, we are starting to understand the "wisdom of the crowd", in fact google has used this in its search algorithms for a while, but there are even more powerful tools that enable collaboration on an unprecidented scale, sites such as wikipedia are classic examples of this.

With respects to the scientific community, by its very nature, it is a dynamic exchange of ideas and theories. unlike other debates we encounter, such as religion, politics etc... there are not the same intitutionalised barriers to callenging the status quo. If a scientist or group of scientists have a theory, this will be critiqued by many other scientists who will then be able to repeat either the same experiment and validate the findings, or disprove them. Even if a scientist or group of scientist start to hold some political sway, and spread inaccurate theories for a time, eventually someone will be able to challenge the theory, and a better theory will prevail.

So where does that leave us with the environment debate. Well, my personal feeling is that the premise that us 6 billion humans are having a noticable impact on the world we live in has been kicking around for long enough now, and the vast majority of scientists who are working in the field seem to agree on some of the big ticket items that constitute climate change. What the results might be are admitedly speculation. Educated speculation, but speculation none the less. However, I think that we can no longer use the ignorance is bliss approach to keeping the status quo. We have to realise that systematic change is essential if we as a species want to continue to maintain the quality of life that we currently do. Even more so if we want to take any significant amount of non-human species into future. This then becomes a battle that sometimes needs to be fought on a political level, and sometimes on a personal level. So Darren, next time I'm bending your ear about how stupid it is to be logging our water catchments for wood chips in the middle of a drought, and you slip seemlessly into that glazed over look, at least you'll know where I'm comming from.

Technorati and claiming your blog

2006-12-03T16:51:00.000+11:00

I started playing around with Technorati the other day and decided to "Claim my blogs", just to see if anyone else was linking to me. I have two blogs, this one which is my geek blog, and another one that I reserve for my thoughts on politics/art/etc... anything that's not technology related. To my surprise I found quite a few people link to this blog, but to mt disappointment, found that I had absolutely NO links to my other blog. I guess that's no surprise really considering I have never been really active in promoting my other blog. So I thought I'd actually attempt to add some technorati goodness by putting in a shameless plug on my popular blog for my less popular blog.

Musings of a Morbid Mind.

No prizes for guessing that aliteration is one of my favourite literary devices.

Development V's Production Environment

2006-10-28T17:51:00.000+10:00

An interesting post on software environmental issues by Mitch Denny caught my attention today, not only because he sent a link around on our internal tech list, but because it's something that I think continually trips up a lot of developers (myself included). Mitch basically argues (and I agree) that development should be done in the same basic environment as the production system, he plays this off against infrastructure people who tend to want to keep a gap between production and development for safety reasons, I mean wo knows what wholes these crazy developers are going to poke in our top-notch security infrastructure.

Most of my experience has been in the shrink wrapped software industry ans as such, there is little point trying to match development and production environments, so in this case I suggest develop on the environment that makes the developers most productive, but then make sure you test, using whatever virtualisation technology you prefer, on ALL platforms in your supported platform matrix.

With enterprise development however, I would say that one should develop in an environment as close as is practicable to the production environment. It never ceases to amaze me the amount of problems that arise because of differences between the development and production environments. Have you ever heard a developer say "gee, it doesn't do that on my machine". The closer you make development environments to the production environment, the less you'll hear this anoying phrase.

The flip side of the coin is that developers need to understand fairly well the differences between their development environment and their production environment. One thing that has bitten me in the past, and a lot of other developers as well is the difference between IIS and the Visual Studio Development Web Server (The web server formerly known as Cassini). There are a number of differneces, and also a number of reasons that developers like to use it over IIS. One major difference which always seems to trip me up is that Cassini passes every requested file through the ASP.Net pipeline, as opposed to IIS which will serve a lot of files normally (ie *.css, *.jpg, *.gif, *.png, *.html, etc...), and only pass the ASP.Net specific files (*.aspx, *.asmx, *.ashx etc...) through to the ASP.Net pipeline. When you first start a project you don't really notice any difference until you think to yourself, "gee, it would be nice to add forms authentication now", and so you put in the standard <deny user="?" /> element and all of a sudden your login page has no graphics and doesn't look anything like how you designed it. This is because Cassini is sending all of you image and css files through the ASP.Net pipeline and they are getting blocked by the Security module because you are not authenticated... yet.

Tags: ASP.Net, IIS

Visual Studio 2005 misbehaving after Automatic updates

2006-10-13T14:08:00.000+10:00

I have spent the morning struggling with Visual Studio. It has been crashing non-stop, but I have got to the bottom of the problem, and am happy to share with the rest of the world.

Visual Studio was happily starting and loading my solution, but whenever I clicked on the test menu and attempted to do something like open the TestView window, VS was crashing. If I tried to open a localtestrun.testrunconfig file it would crash with the following error "Could not open assembly System.Data version 2.0.0.0 .... The system cannot find specified path".

I won't bore you with the process I had to go through to find the resolution, but the problem turned out to be that this morning My machine installed some updates from Windows Automatic Updates. Most of these updates were succesful, but one of them, namely "Security Update for Microsoft .NET Framework, Version 2.0 (KB922770)" had failed with the following error "Error Code: 0x643". after doing a search on this particular error I found the following article http://support.microsoft.com/kb/923100. However, I did not need to follow the resolution outlined there, I simply went to the Windows Update site and manually installed the update.

I hope this helps someone out before they have to spend half a day like I did trying to track it down.

PopupControlExtender inside an EditTemplate

2006-10-10T12:58:00.000+10:00

I found a problem today with the PopupControlExtender (Part of the ATLAS Control ToolKit), that occurs when it is embedded inside an EditTemplate for a GridView Control. I suspect the same problem would occur inside any template field, and potentially for other ATLAS Control Toolkit controls.

I wanted to edit a date field inside my EditTemplate, and so I set about implementing something not too dissimlar to the example for the PopupControlExtender on the Atlas Control Toolkit website. Firstly I prototyped it on a page all on its own, no problems everything worked as expected. Then I attempted to place the same code inside my EditTemplate field. At this point when I attempted to go into edit mode I recieved an error message "Assertion Failed unrecognized tag atlascontroltoolkit:popupControlBehavior". Obviously the necessary javascript libraries were not being downloaded when the page rendered. I assumed this was because of the dynamic way in which the EditTemplate renders.

To fix this problem, I simply placed an empty PopupControlExtender onto the design surface of my page. This ensures the necessary javascript files are downloaded, and when the EditTemplate is rendered, the ATLAS engine knows what you're talking about.

I also encountered a coulpe of bugs with the popup control being wrapped inside an update panel. I managed to fix these by simply downloading the latest CTP of the ATLAS Control Toolkit, (now called the "Microsoft ASP.Net Ajax Toolkit"... sorry guys, this is yet another example of how insisting on meaningful naming of products can really kill any excitement and mistique around it, I'm afraid it just doesn't have the same panache as "Ruby On Rails", and if you really want to speak to the buzzword concsious managers of today, you have to break away from boring naming conventions and inject some creativity into the process).

tags: ASP.Net, AJAX

Reflections on Web Directions

2006-10-09T19:09:00.000+10:00

I have to be the slowest blogger in the IT community. Over a week ago I attended the Web Directions conference. The conference was really good, and has inspired me in a few areas. Many other bloggers have had their say on the conference, but I may as well have mine, just for the sake of redundancy.

One of the common themes from both the Accessibility speakers (Gian Sampson-Wild and Derek Featherstone) and the User Experience speaker (Kelly Goto) was user testing, and not just asking the user what they think. Kelly Goto's quote was 'We listen to what the users "didn't say" and observed what they did'. I have also been challenged in the area of accessibility, I think the quote for me came from Derek Featherstone, and that is "The web is Accessible by default, we make it in-accessible". It has inspired me to go have a look at the way I develop, and the bad habits I've gotten into. The truth of the matter is that it is not really that much effort to get into good habbits that make web sites more accessible.

I really liked Jeremy Keith's AJAX sessions. The first session started out a little basic, but become more interesting in the last half. In the second session Jeremy discussed a technique he called "Hijax" which is aimed at ensuring accessibility, and support for down level browsers. I am looking forward to seeing how well I can apply his techniques using ATLAS. While on the topic of libraries, Jeremy did make one statement that I'm not sure I can agree with. He said that he didn't believe in using thrid party libraries for doing AJAX, firstly becasue AJAX wasn't that complicated and secondly because if something goes wrong in the library as a developer you'll need to be able to fix it. I almost agree with his first statement, but even still, I am a big fan in NOT re-inventing the wheel. If there's a library that has a great ranking control for example, and they've coded it so that it works across all browsers in your supported browser matrix, then there is a lot of testing and coding that you can potentially avoid. There are always bugs in any software, and AJAX APIs are no exception, the skill of a good developer is to be able to use an API in such a way that they can workaround any bugs in the underlying API, I've lost count how many times I've had to do this myself. Also, as a Winforms developer, I am extremely greatful that I do not have to write Win32 anymore, and I'm sure those of you who have written Win32 would agree with me. Having said that, his "hijax" mechanism of progressive enhancement is really cool, and my current goal is to go through all the ATLAS controls (the API I'm currently using) and see how I can apply this technique to them.

I went along to John Allsops talk on microformats to hopefully pickup anything I'd missed from the first time I heard it, and was inspired all over again.

One minor thing I think the organisors can improve on is something I saw at the Tech-ed conference, the "re-charge desks". The tech-ed organisors had desks with a series of power boards that people could plug their lap tops into between sessions. This would have been really good for me as my laptop is now a year and a half old, and my battery is showing its age, it wasn't even lasting 2 hours.

Worth mentioning:

Kelly Goto made reference to an example of an interesting User Experience project called Datelens. It is a callendaring visualisation which plugs into outlook (nothing to do with the web at all). I downloaded it and it's quite interesting, definately worth a look. Also it is written in .Net 1.1, which must have been quite an acheivement, and I think with WPF just around the corner, there will be a lot more of these types of user experience applications about.
The inaugural McFarlane proze for Excellence in standards based web design was awarded to http://www.museum.vic.gov.au/caughtandcoloured/. This site is really worth checking out. It is especially interesting to me because it is written in ASP.Net demonstrating that it is possible to write high quality standards compliant accessible web sites using ASP.Net.
- The last speaker of the conference was Mark Pesce (inventor of VRML, explained some of his concepts of social software, and described a project he worked on that attempted to aggregate your social behavior patterns and create social network models simply by using the ability of a blue tooth enabled phone. I thought there’d be some people at readify quite interested in this sort of stuff. http://relationalspace.org/

tags: wd06

ATLAS Data Applications and Down-level support

2006-09-17T18:13:00.000+10:00

I must confess that aslthough I am excited by the whole AJAX phenonenon, and the rich client experience that it provides for the majority of desktop browsers, I sometimes feel concerned about those that for whatever reason aren't able to experience the full beauty of a javascript enabled client. There are two main groups of people that are missing out

Users with certain accessibility needs (often browsers and tools designed for people with disabilities have to run without the javascript support we usually take for granted) and
PDA users (This is an increasing market, and to date only 1 browser has implemented any kind of javascript engine for devices, Opera.

So I am always looking for ways to enhance these users experience.

My AJAX toolkit of choice (more through accident than intent) is ATLAS (Microsofts library). ATLAS contains a data enabled control called a ListView which is quite powerful. It works on a client side JavaScript based object model and the best way to get to know it is to read the data section of the ATAS documentation.

So the idea is, get a funky ATLAS based website working, and then attempt to create a version of that website as quickly as possible that can be used by people who can't (or won't) use javascript (similar to what google does with gmail).

I decided to make the sample as simple as possible without loosing anything along the way. I decided to create a Shopping List, that simply contains a quantity (double) and an item (string).

The bueaty of the ListView in ATLAS is that it can use the System.Component model similar to other databinding in ASP.Net 2.0. So the first step in creating a webservice that derives from Microsoft.Web.DataService (refer to the ATLAS documentation for more details). I then create my ListView templates and hook it all up using XML-Script, and pretty it upwith any CSS I may care to add. The result can be seen here (Please forgive me I'm not a graphic artist). Once this is done, I needed to create a Non-ATLAS version of the same page. As my webservice already uses the System.Component model, it was extremely easly to add a GridView control, select an Object DataSource and wire it up, and away it goes. All in all it took me abou 10 minutes to create the NON-Atlas page. It would have taken me another 2 minutes to enable things like edit/delete, but for the moment I wanted to keep it the same as the ATLAS version, this may be the subject of a future post. The only significant difference between the two pages is the presence of a select link on the NON-ATLAS enabled version whereas the ATLAS version manages the select operation simply by handling the mouse click.

Once you have these two pages it is simply a matter of choosing your favourite way to direct users with different broswser capabilities to the correct site.

ajax gotcha for new players

2006-08-20T01:05:00.000+10:00

Being fairly new to the "Brave New World" of ajax, this week I fell into a fairly common trap that gets new players. I studiously looked around for the best practice implementation of making an http request using javascript and came up with the following javascript function which allows the user to make the http request, specify a callback function, and even goes the extra mile of implimenting timeouts (I guess the only other thing I really should do is implement an OnErrorCallback).


function MakeRequest(url, timeout, 
    onCompleteCallback, onTimeoutCallback)
{
    var completeCallback = onCompleteCallback;
    var timeoutCallback = onTimeoutCallback;        
         
    var xmlHttp=createXMLHttpRequest(); 
    
    if (xmlHttp)
    {          
        xmlHttp.onreadystatechange = function()
        {                
            if (xmlHttp.readyState == 4)
            {            
                window.clearTimeout(timeoutId);
                if (xmlHttp.status == 200 || 
                    xmlHttp.status == 304)
                {                
                    completeCallback(
                        xmlHttp.responseText);
                }
            }
        };
                                 
        xmlHttp.open("GET",url,true);
        
        var timeoutId = 
            window.setTimeout(function()
        {        
            if (callInProgress(xmlHttp))
            {
                xmlHttp.abort();
                timeoutCallback('timeout');
            }
        }         , timeout);
        
        xmlHttp.send(null);        
    }
}

function callInProgress(xmlHttp) 
{
    switch ( xmlHttp.readyState ) 
    {
        case 1, 2, 3:
        return true;
        break;

        // Case 4 and 0
        default:
        return false;
        break;
    }
}


function createXMLHttpRequest() {
  if(window.XMLHttpRequest) {
    try { 
      xmlHttpRequest = new XMLHttpRequest();
    } catch(e) { return null; }
  } else if(window.ActiveXObject) {
    try {
      xmlHttpRequest = 
        new ActiveXObject("Msxml2.XMLHTTP");
    } catch(e) {
      try {
        xmlHttpRequest = 
          new ActiveXObject("Microsoft.XMLHTTP");
      } catch (e) { return null; }
    }
  } else return null;
  return xmlHttpRequest;
}

I then wanted to call a .Net HttpHandler, and lets just say for arguments sake that the htp handler looks like this


public class Handler : IHttpHandler
{
    
    public void ProcessRequest (HttpContext context) 
    {
        System.Threading.Thread.Sleep(6000);
        context.Response.ContentType = "text/plain";
        context.Response.Write(Guid.NewGuid().ToString());
    }
 
    public bool IsReusable 
    {
        get 
        {
            return false;
        }
    }       
}

so the call from javascript would simply look something like this



        MakeRequest(
            'Handler.ashx?cookie=abcd&another=xyz', 
            20000, onRequestComplete, 
            onRequestTimeout);       
        
        function onRequestTimeout(response)
        {
            alert('timeout');
        }
        
        function onRequestComplete(response)
        {
            alert(response);
        }

The problem as I found out through good old wikipedia relates to the internet explorers caching of the "GET" http command as described in this article on HMLHttpRequest.
As discussed in the article, there are a number of ways around this, switch to using POST or ensure that you switch the caching off in the http headers. I chose the former.

Automatic updates + Automatic Reboots

2006-08-13T12:29:00.000+10:00

Is it only me or does anyone else find this really frustrating.

I got this in the middle of reading my email this morning, and I really didn't want to reboot in 5 minutes. I would have quite hapoily rebooted in half an hour or so when I'd finished what I was doing. The fact that the restart later button was there but disabled just added to the insult.

I think from a layer 8 perspective, Microsoft need to be really careful when forcing reboots. If this happens too frequently, it may be frustrating enough that people actually turn Automatic Updates off, making their systems progressively less secure and vulnerable.

Web Directions

2006-08-12T11:06:00.000+10:00

On thursday night I went along to a Web Directions preview night where 2 of the speakers from the upcomming Web Directions conference Ben Barren from gnoos and John Allsop from westciv were giving previews of their presentatoins at the conference. It was a good night, I particularly enjoyed Johns talk on microformats. This is a topic that I have been interested in for a while now, and may have more to say about it later.

Unlike other events I attend, where I know a fair few people, this event I only found one person I'd met before was Nigel Watson, an Architect Advisor from Microsoft. So it was good to make new contacts and expand my circle beyond the sphere of the Melbourne .Net developer and SQL Server sphere. I got to meet Cameron Adams whose book "The Javascript Anthology" I am currently half way through, and also met the people from westciv who write the best damn web standards online training courses in existance.

When I first heard about the web directions conference, I thought that as much as I'd love to go along, I probably wouldn't be able to attend given that I was already attending Tech Ed at the end of August, and I'd be pushing it a little if I asked my employer to pay for another conference and let me have the time off so close after Tech Ed. As the saying goes, "the best layed plans of mice and men gang aft aglay", there was a lucky door prize drawn at the end of the night, which was a free ticket to attend the Web Directions conference. Guess who won........ So now all I have to do is convince my employer to let me have a couple of days off to attend..... that shouldn't be too hard.

Stepping through ATLAS Javascript files

2006-07-13T17:40:00.000+10:00

I'm trying to debug an application I'm writing using ATLAS, and I'm wnating to know why my attempts to make browser client magic happen are resulting in java script errors. Fortunately Visual Studio provides a level of debugging for java script files which means that if you're in debug mode when a java script error occurs, you'll automatically be taken to the point in the code where the problem occured and given some useful things like a stack trace and a variable watch window. You can also set break points in your own code, however, if you have registered the scriptsw using an Atlas Script manager as is advised when writing ASP.Net applications

<atlas:ScriptManager ID="ScriptManager1" runat="server" >
<Scripts>
<atlas:ScriptReference ScriptName="AtlasUIDragDrop" />
</Scripts>
</atlas:ScriptManager>

you will not be able step into any of the ATLAS libraries you happen to be using as the debugger complains that the source files are not available. This is because of the way the ATLAS scrip manager embeds the java script.

today I found myself thinking that it would be handy to be able to step into these files to find out what's going on prior to the error.

To solve the problem

Copy the necessary java script files from the install location into your project and mark them as "Embedded Resource". In my case Atlas.js and AtlasUIDragDrop.js

Comment out or remove the Atlas Script Manager component

Add the script files as you would add any normal java script file (ie with the <script> tag)

Once you have done this you should be able to step into the ATLAS java script files to your hearts content, but be warned you might find it a little scarey in there.

The only problem with this is that it won't work for Server Side Atlas controls, such as UpdatePanel or Extender controls, but if you're not using them, or if you can comment them out while you debug stuff, then you're set.

Google Maps and sustainable transport

2006-07-06T08:59:00.000+10:00

I have found a great site that is a mashup of google maps. http://www.bikely.com. The idea is you can search the user entered database for the best way to get between various points by bike. The website also allows you to sign in and create your own bike route so you can share your favourite rides with others. You can even put comments at each point that appear as pop-ups on the map. I'll have to start riding my bike more so that I can contribute.

Browser Version hell

2006-06-27T13:01:00.000+10:00

Those familiar with the concept of DLL hell and have experienced it first hand often point to web based applications as the savior for such deployment issues, and to an extent they're correct. However, as the push for a richer client experience from web applications grows, and as the technologies that enable it mature, DLL hell is being replaced by Browser version hell. We are all aware that Internet Explorer and the Mozilla based browsers (firefox) have their own peculiarities, but even beyond that are the versions of these browsers. In fact, IE 6 has been particularly slammed for some of its, shall we say "non-standard" behaviour. As a result of this I have been a big fan of firefox that is considerably more standards compliant. This has lead to web designers targeting firefox first, and then tweaking sites as needed for other browsers.

For the past 6 months or so I have been neglecting firefox a little in order to try out the all new IE 7 in its various beta forms. IE 7 is supposed to be far more standards compliant, amongst adding other features users of firefox and other browsers take for granted. So as a result, I haven't updated firefox for a while, and using firefox version 1.0.7 I was testing out a new Atlas based control called the collapsible panel control, and the bad news is, It doesn't work. The area that should display when the panel is expanded is displayed completely blank. Upgrading to version 1.5.04 fixes the problem completely. Now admittedly the Atlas Control Toolkit is very much in a pre-release state, but it does raise a very pertinent question.

Where should web developers draw the line and say I'm not going to support versions of browser X before version Y, and how should web sites indicate that a user should upgrade their browser? Also, how many different versions of browser X should we be realistically expected to test under? This I believe is different to the idea of degrading nicely (ie what should happen if a browser doesn't support Java Script or only supports a limited subset of Java Script). All I can say is that I hope the team at Microsoft fix the problem with the control for firefox Version 1.0.7.

Web Application Projects and SQL Express Data Sources

2006-06-26T19:09:00.000+10:00

I have been struggling today with some SQL Express related data issues. I have been trying to use a combination of Web Application Projects and ATLAS, as I am convinced that Microsoft have completely stuffed up with their "Web Site" style projects, I really don't want to use them. So to do this you simply create a new Web Application Project, and then copy all the necessary settings from the sample ATLAS web.config file into your own web.config file, and ALL is fine... that is until you want to do something with SQL Express. Regardless of ATLAS, the standard approach is to add a new item to the project and select a SQL Database. This uses SQL Express in a mode called "User Instances". This is designed to allow you to treat the databases primary file (mydatabase.mdf) as though it is just that, a file (impressive if you know what's really going on under the hood and understand how SQL Server normally works), and this is how all the Microsoft demos go, except that they use Web Sites instead of Web Application Projects. The theory is then to add tables, and create a DataSet off these tables. So I thought it would just be a simple matter of creating a Web Application Project, and then following exactly the same logic. WRONG!!!!!!

My first efforts yielded the following error

Description: An error occurred during the compilation of a resource required to service this request. Please review the following specific error details and modify your source code appropriately.

Compiler Error Message: CS0122: MyWebApplicationProject.Properties.Settings' is inaccessible due to its protection level

This is because the default way of storing the connection string is to use the Properties folder and create an entry in the Setteings.Settings file. In this case there is a class called MyWebApplication.Properties.Settings. This has a protection level of "internal" meaning that only code from within the same assembly can refer to it. The code attempting to access it is NOT in the same assembly so it fails compilation.... DOH.

Next step was to attempt to rework it so that it got the connection string from the web.config file. After correcting all the places where I found a reference to the connection string, I received the following error.

Exception Details: System.Data.SqlClient.SqlException: An attempt to attach an auto-named database for file C:\Projects\AtlasApp\AtlasApp\App_Data\App_Data\Tasks.mdf failed. A database with the same name exists, or specified file cannot be opened, or it is located on UNC share.

Better, at least this time it's compiling. After spending entirely too much time attempting to figure out why SQL Server User Instances were failing, I eventually re-read the error message, and noticed the path it was attempting to access. C:\Projects\AtlasApp\AtlasApp\App_Data\App_Data\Tasks.mdf

Notice the two App_Data directories. Yep, so simply exclude the App_Data part of the path in the connection string and away it goes, everything now works.

The reason I write this article is because I found very little documentation on the problems. Problems which I thought would be occurring quite frequently if people are using Web Application Projects, but alas the only mentions I have found are a recognition by Microsoft that there are problems, and a vague promises of a white paper on the issue from Microsoft.

In conclusion, I've had quite a frustrating day, and I think Microsoft still have some work to do on Web Application Projects.

using a custom database for persisted SQL Server ASP.Net session state

2006-05-26T15:53:00.000+10:00

I am in the process of upgrading a major project from ASP.Net 1.1 to ASP.Net 2.0, and my ideal is to have 2 versions of the web application able to be run so that I can compare what's going on in the .Net 2.0 version to what should go on according to the .Net 1.1 version. This doesn't seem too difficult, just use a different web folder right? Well, this would be the case except that we're using SQL Server persisted session state and although they use a different schema between .Net 1.1 and .Net 2.0, the default database name is the same "ASPState". After a bit of fiddling around I eventually solved the problem.

Step 1 run the following command line from the appropriate .Net 2.0 directory replacing ASPState_2_0 with the database name you wish to use

aspnet_regsql -S localhost -E -ssa
dd -sstype c -d ASPState_2_0

This creates the session state database with the appropriate schema in the specified database name instead of the default.

Step 2 modify the following line in the web.config file accordingly

<sessionState mode="SQLServer" allowCustomSqlDatabase="true" stateConnectionString="tcpip=127.0.0.1:42424" sqlConnectionString="data source=localhost;Initial Catalog=ASPState_2_0;Trusted_Connection=Yes;" cookieless="false" timeout="15" />

Step 3 Manually give the ASPNet user (or whatever user you are running the ASP.Net web application as the appropriate priviledges on the database tables in the newly created session state databse.

MEDC 2006

2006-05-26T09:01:00.000+10:00

Just got back from the MEDC 2006 conference where I was lucky enough to recieve an I-Mate SP5 simply for attending.

The sessions were all very interesting, my personal favourite was probably the session on performance in the compact framework.

The other interesting thing I learnt is that people are actually reading my blog. Well, if that's the case I might have to buck up my ideas and actually put some half decent content on.

A new and exciting project

2006-05-24T09:32:00.000+10:00

This week I started a new and very exciting project. I can't really give away all the details as the project is still in the planning phase, but what I can say is that it will be a Web 2.0 project. Now I know Web 2.0 can sound like a bit of a buzz word and as such looses meaning, but I think this article by O'Reily explains Web 2.0 in a way that cuts through most of the marketing hype.

The Web 2.0 concepts that we are thinking of using in this project involve the idea of Mash Up's of a few different services, the idea of collaboration and collective intelligence, technologies such as ATLAS (AJAX for ASP.Net) and the concept of the quick release cycle.

It is interesting that I have ended up on this project. I must be the least experienced ASP.Net developer at Readify, and the last time I did anything commercially for the WEB (almost 2 years ago now), was firmly rooted in Web 1.0 mindset. Since then I have been firmly stuck on the Rich Client side of the divide writing Pocket PC and Desktop style applications. The team lead on this project has suggested that this is probably a good thing as we can take my experience designing and working with rich UI's and not be limited in my thinking of what is capable on the web. We are all getting really excited about the potential of this project, and I will give more details as the project progresses.

To Train or Be Trained... That is the question

2006-05-08T10:56:00.000+10:00

Over the past 15 months I have been consulting to QSR and it has been a fantastic experience. The only issue has been that since I've been there I have worked exclusively on a winforms .Net 2.0 application. This has meant that my ASP.Net 2.0 skills were fairly rudamentry, and weren't improving beyond the tiny bit of time I've been able to allocate to professional development on the bus on the way to work.
So I decided to do the Readify Professional .Net which has been recently re-vamped for .Net 2.0. So I rang Chris Hewitt who was instructing the course to tell him of my intention. Chris's paraphrased reply was something like ... "You've been using .Net 2.0 for over a year now, you know more about .Net 2.0 than I do, you can teach some of the course".

So this is how I ended up teaching three modules of a course I was originally intending to sit. The 3 modules I taught were Generics (one of my favourite .Net 2.0 features), Click Once (on Chris's suggestion) and refactoring. It is my goal to to be trained up to a point where I can teach the course myself, and it was good sitting under Chris who has extensive teaching experience. I have always enjoyed teaching ever since I supported myself through uni by tutoring high school maths and physics, and I know it is one of the best ways to ensure your own grounding of the material. The thing that scares me the most is questions. I know I shouldn't be scared, we aren't expected to know everything, and in the rare cases when Chris didn't know the answers, it proved to be a real journey of discovery for allpf us as we attempted to find the answers to these questions.

Click Once Deployment Gotcha

2006-05-02T23:02:00.000+10:00

In attempting to prepare for a module I am teaching tomorrow, I am putting Click Once through its paces, and I'm finding there are some strange issues. I have been struggling with an unusual error for half the night.

I am Publishing my files to a local site, then openning up internet explorer, navigating to the site and then attempting to Install the application. The application contains a 4 files that require downloading, and it uses SQL Server Express. The Install starts out fine, it then decides it needs to momentarily open a browser window. Now my default browser is Fire Fox, and it decides to open this. At this point firefox decides to prompt me for a username and password. Why I don't know, but seconds after typing in my credentials, it brings up an error saying that it cannot download the application as there are missing files. The install fails.

I eventually figured out that if I change my default browser back to be internet explorer, the problem goes away. Hope this helps someone.

SQL Server Express with Advanced Services

2006-04-24T19:33:00.002+10:00

As an avid user of SQL Server Express, it would be remiss of me not to herald the release of the much anticipated SQL Server Express Edition with Advanced Services with a blog post. This adds SQL Server Reporting Services and SQL Server Full text search as well as its very own version of management studio into the mix which now gives it a distinct advantage over its predecessor MSDE.

IMHO the only thing that is now missing from SQL Express is SSIS. I think that adding SSIS to SQL Express would allow people to enter more data quicker and would therefore come up against the size and performance related restrictions of SQL Server Express, and would cause people to upgrade to other SKU’s of SQL Server sooner.

SQL Server 2005 installation problems on various CPU's

2006-04-11T10:26:00.000+10:00

As I have become the unlikely SQL Server 2005 "expert" at the company I am currently consulting to, I have been investigating a wide number of installation issue with SQL Server Express. There are a few main categories these problems fall into.

1. Installation of beta versions of SQL Server 2005 or the .Net Framework 2.0 (solution uninstall all beta versions of SQL Server 2005 or .Net 2.0, and if you've installed beta versions of Visual Studio 2005, see http://msdn.microsoft.com/vstudio/support/uninstall/default.aspx)
2. The CPU is NOT fully Pentium III compatible
3. Various issues ranging from disk space to memory to OS service pack level etc...

In this post I want to deal with problem #2, The CPU compatibility problem.
There seem to be a number of CPU's that claim some kind of Pentium III compatibility, and as the hardware and software requirements for SQL Server 2005 state, Pentium III is the minimum hardware requirement.

However, for whatever reason, the SQL Server installer does not check CPU compatibility too thoroughly as we have seen a in a number of cases that the installer will issue a Hardware warning, but happily allow the user to continue to attempt to install SQL Server 2005 but will in the very final stages of the install refuse to start the SQL Server service. I'm not sure of all of the reasons why this error potentially occurs, but I know 1 definite problem is that SQL Server insists on the cache prefethching instructions of the Pentium III CPU. This is missing in a fair amount of CPU's that claim compatibility with Pentium III's. I have asked the guys at Microsoft about the minimum hardware warning, and they confess it is a problem, and are reviewing whether they are going to be more specific in the future, however, I think there needs to be some kind of list of all the different CPU's that one might reasonably expect to be able to install SQL Server 2005 on, but don't seem to have the ability to. So here it is, or at least the results of my research, if anyone else has seen similar problems, please leave a comment with the CPU specs and any links to discussions about the problem on various forums.

Via Eden - http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=304355&SiteID=1

500 MHz AMD Family 5, Model 8, Stepping 12 processor (AMD-K6-2) - http://www.sqljunkies.com/Forums/ShowPost.aspx?PostID=167¬ification_id=421614&message_id=421614

VIA Ezra 800Mhz processor - http://www.sqljunkies.com/Forums/ShowPost.aspx?PostID=167¬ification_id=421614&message_id=421614

Microsoft Virtual PC For Mac OS 7.0.2 (This emulates a Pentium II) http://www.microsoft.com/communities/newsgroups/en-us/default.aspx?dg=microsoft.public.mac.virtualpc&tid=90715b2e-77ba-4f8a-a189-daee29467ff0&cat=en_US_595a5881-be4a-4db4-97b0-5f3a78602e23&lang=en&cr=US&sloc=en-us&m=1&p=1

Transmeta - http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=17395&SiteID=1 and http://lab.msdn.microsoft.com/ProductFeedback/viewfeedback.aspx?feedbackid=cee0915a-6039-4ca6-aa0c-1b76285d4ac3