Wednesday, October 10, 2012

Getting the correct versions of .Net Framework for WinDbg

To debug a minidump of a .Net application created by Windows Error Reporting, it is essential that you have the correct version of the following 3 dll's
sos.dll
mscorwks.dll
mscordacwks.dll
The correct version being the version of the .Net framework that the user was running at the time the minidump was generated. For example, if the minidump was created using the framework 2.0.50727.4971, then you need the 2.0.50727.4971 version of the dll's mentioned above to debug this. Chances are this is NOT the version you currently have on your machine, it may not even be a valid version for your OS (XP, Vista, 7, 8) or architecture (x86, x64). As far as I'm aware, there is no central repository where you can go an grab the correct version of these libraries from, so one has to attempt to find them the hard way.
The .Net framework is initially deployed with our application, however, it is regularly updated, once installed, by Windows Update, to ensure that security vulnerabilities and major defects are patched. If your application is only installed on a single server (or server farm), this is not really a challenge, however, if your application is installed as a desktop application on nearly quater of a million PC's world wide, finding the correct version of the framework requires some sleuthing skills, and a bit of luck. The first step is to try and find the Microsoft support knowledge base article that describes the update. To do this I've found the following query typed into google, usually gets the result you want.

.Net framework 2.0.50727.4971 site:support.microsoft.com

It is usually the first hit, but it may be further down. Validate this by searching the resulting page (Ctrl+F) for the framework version (e.g. 2.0.50727.4971) and make sure SOS.dll one of the binaries with this version number.
clip_image001
Go to the download information section of this website and click on the provided link. This should take you to the Download Center page where you should see the following options
clip_image002
Click to download the architecture you require. This will download the .msu file for the update. You can use the command-line Expand utility to extract the files in this update package.
expand.exe -D .\Windows6.1-KB2604114-x86.msu
Will list all the files in the package, in this case there are 4,
.\windows6.1-kb2604114-x86.msu: WSUSSCAN.cab
.\windows6.1-kb2604114-x86.msu: Windows6.1-KB2604114-x86.cab
.\windows6.1-kb2604114-x86.msu: Windows6.1-KB2604114-x86-pkgProperties.txt
.\windows6.1-kb2604114-x86.msu: Windows6.1-KB2604114-x86.xml
You want to extract the Windows6.1-KB2604114-x86.cab file to do this use
expand.exe -F:Windows6.1-KB2604114-x86.cab .\Windows6.1-KB2604114-x86.msu .\
You can then use expand.exe to interrogate and extract the contents of this .cab file.
To extract the files you want use
expand.exe -F:sos.dll .\Windows6.1-KB2604114-x86.cab .\
expand.exe -F:mscorwks.dll .\Windows6.1-KB2604114-x86.cab .\
expand.exe -F:mscordacwks.dll .\Windows6.1-KB2604114-x86.cab .\
Each of these statements will usually result in 2 folders named something like the following
x86_netfx-sos_dll_b03f5f7f11d50a3a_6.1.7600.16992_none_e878512ab7239f09
You'll need to check both dll's and make sure you get the version you are after.
Simply copy these into a common folder, and you can then use the following windbg commands to set the exepath and load the correct sos.dll
.exepath+ C:\MyFrameworkFolder\fwx86_2.0.50727.4927
.load C:\MyFrameworkFolder\fwx86_2.0.50727.4927\sos.dll
Happy debugging.

Monday, September 17, 2012

Telstra’s Transparent Proxying

I am finding Telstras Transparent proxying a nightmare at the moment. I am trying to configure a Windows Azure SQL DB instance I have set up on a trial account I got from TechEd. Azure SQL database security is tied to your IP address. You can click “Manage” from the portal window to add your current IP address to the firewall rules, but then thanks to Telstra’s transparent proxying, when you connect with SQL Server Management Studio, you get a different IP address, and consequently can’t access SQL Server.

Anyone know of a solution to this, short of getting a fixed IP address and opening up the firewall to the world?

Friday, September 07, 2012

Thoughts on Gina’s $2 a day comment

I think there is a point where the ruling class become so arrogant and full of their own self importance that the “plebs” see their only logical course of action is full scale revolution. I don’t think we’re there yet, and as history has taught us, revolutions rarely succeed in doing more than replacing the head on the coin, but with Gina Rinehart’s latest comments, I feel we have inched one step closer to it. Indeed there are people in West Africa who will work for $2 a day. Many West African nations are struggling economically, and desperate people will do desperate things. Perhaps the global economy does need some re-adjustment, and maybe just, the “invisible hand” of the market can do something to address the great injustices of the world, but for Gina Rinehart to make the inference that Australians need to take severe pay-cuts just because her precious mining business is in danger of slowing down, makes it clear just how detached from reality she has become. You can almost hear “let them eat cake”. It is not the responsibility of the working class to fund the ruling classes lifestyle. If you can’t make your business profitable while being ethical to your workers and paying them a liveable wage that takes into account cultural and social standards, then you don’t deserve to be running a business. Perhaps if Gina was to spend time in West Africa working for $2 a day, she may have a little more sympathy. Even if she was forced for a few months to exist on an Australian un-skilled labourers wage, she may get the picture.

Sunday, November 13, 2011

The Slippery Slide

I was raised as a fundamentalist Christian. Until the age of about 25, I believed the following facts to be immutable.

  1. There is a metaphysical all knowing and perfect being who exists outside of time and space, who for want of a better name is called God.
  2. This God being, created the entire observable universe including a species of cognisant beings in his image who at some point in their "time based" existence started referring to themselves as homo sapiens. These beings were created with a mortal physical presence and an immortal soul.
  3. This God being, spoke to some of these cognisant beings throughout this time based history, that were deemed worthy to accurately record all of his various thoughts and actions over the course of a few thousand years. These writings were then collected by a committee of these cognisant beings around 300 ACE who were doing the will of this God being. This collection of works went by the name "The Holy Bible".
  4. This God being made up some elaborate rules for these cognisant beings to gain this Gods favour and be eligible to co-exist with this God being after there mortal physical presence had expired, in a really nice place. The alternative to this really nice place was a pretty horrible place, and those who didn't follow these elaborate rules would be sent there. These elaborate rules were spelled out in this Holy Bible book. Oh and
  5. This God being has a penis... (A freakin' huge one).

I chose the word "fundamentalist" in my opening sentence very specifically. There is a quintessential difference between someone who believes

  1. The Bible is the written word of God and infallible, and someone who believes that
  2. The Bible is a book of collected works written by humans who were fallible, but had certain insights into the divine at a particular point in history.

There are many aspects in which this distinction plays itself out. One we see commonly, is the biblical literalists insistence on attacking the scientific theory of evolution, and insisting, against all evidence to the contrary, that the earth is less than 10,000 years old. The reason Biblical literalists cannot, and will not accept that the creation story is anything more than a myth for the benefit of an illiterate desert tribe to take solace in, is because they argue that this denial in the literal truth of the opening chapter of the Bible creates a slippery slide that will hold the rest of the Bibles message to critique, and will ultimately invalidate (in their minds) the undeniable truth of the word of God. It has very little to do with the psuedo-science of intelligent design. People who fall into category b however, (those I would refer to as liberal Christians), see no problem with juxtaposing the creation myth as a beautiful and metaphorically elegant story of the supernatural origins of the universe against the scientific theory of evolution. I myself went through a phase of believing that the Bible wasn't meant to be a modern scientific dissertation, but rather in regards to the creation story was meant to provide sufficient information for people in a pre-scientific era to satisfy the ontological question, and move beyond this to more important aspects of how to live a good life. While I was still able to accept the rest of the dogmas of the particular brand of Christianity I was involved with at the time, this event marked my slow progression from fundamentalist Christian to liberal Christian, which eventually culminated in my current position of Atheism. But there was much more to this progression than simply taking the creation story metaphorically.

I've spoken with a number of ex-fundamentalists. Some have, like me, gone to almost the polar opposite of atheism. Others have chosen the less extreme agnostic route. Some have found solace in some form of deism, while others prefer to hold to their theistic understanding of the world but chose a more liberal approach to their Christianity. Many of these people, however, will have some key thought, argument or idea that has eventually broken the continuum of the cyclic logic that is necessary to maintain a fundamentalist belief system. What I wish to do in this essay is to explain what that argument was for me, and how I was finally able to break out of the fundamentalist mind-set. Note to any fundamentalists who might be reading this essay, before you start formulating your killer cyclic argument that will in your mind destroy my "sole" objection to your fundamentalist belief, just be warned that this is not my "sole" objection, but merely the one that I deem responsible for enabling me to jettison my dogmatic and limited world view.

Language

Before I go on, I need to say something about language, and my understanding of it. I have always had an interest in languages. If you haven't figured out by now, English is my native language, and despite attempts at learning a few others, I have to concede that I am not yet bi-lingual, although it remains one of my life goals to become so. In high school I did a few years of Italian, not that I remember much of it now. I have had a number of attempts at learning Spanish, and it is currently the language I am most fluent in aside of English. I can actually hold decent conversations in Spanish, and understand a reasonable amount. I even attempted to learn Greek at one stage. Very early on in my investigation of the concepts of language, and the ways in which languages differ, I started to feel very overwhelmed with the way in which the language we speak, and the way in which we use language can shape our subjective experience and understanding of the world. This can be demonstrated using a very simple example such as the following.

In English we would say

"The red shirt"

The equivalent Spanish translation would be

"La camisa roja"

Even in this very simple 3 word example we notice two very astounding things. Firstly the word ordering is different. In English we have the definite article "the" followed by the adjective "red" and finally the noun "shirt". In Spanish we have the definite article "la" followed immediately by the noun "camisa" and then finally the adjective "roja". The second and even more significant difference is that in Spanish nouns have a gender whereas in English they are gender neutral. The word for shirt in Spanish "camisa" ends in an "a" making this word feminine, and forcing the usage feminine definite article "la" instead of "el"(masculine) as well as the feminine form of the adjective for red "roja" instead of "rojo".

These differences may seem small, and perhaps just an annoying little peculiarity if you are trying to learn a enough of Spanish so that you can order a meal at a restaurant when you go on holidays. However, if you look a little deeper, it raises all sorts of questions around how the world is viewed by the native speakers of these different languages, and how the language you use to think in shapes your subjective reality. The ordering of words is important to the emphasis we place on things, and concepts. Attributing a gender to nouns, or not changes your whole understanding of that concept, and forms different associations in the neural pathways of the brain. Admittedly, in the very simple example I give here, we a reasonably certain that we are talking about the same thing, but this is an extremely simplistic example. It doesn't take too long into trying to explain more detailed concepts that you very quickly see a difference in the ways in which people with different languages understand and perceive the world. This fact is demonstrated quite profoundly by recent scientific research that shows how our concept of languages affects the colours we perceive and how well we perceive them.

Spanish and English aren't even all that far apart in their linguistic origins. Extrapolate these differences between say Hindi or Mandarin and English, and the mind can only begin to imagine how differently these people must perceive the world and relate to it. Added to this, our language is constantly changing (I would normally use the word "evolving", but I didn't want to alienate any creationists that might be reading). Just reading a play by William Shakespeare will demonstrate just how much the English language has changed in the past 400 years. It has changed so much that learned Academics debate furiously about the meaning of certain words and phrases that subtly (or sometimes not so subtly) alter the understood meaning of the plays.

The Holy Book

This brings me to the main intent of this essay. The argument that caused me to break the cyclic logic of fundamentalism, and allowed me the freedom to intellectually assess the evidence for myself and more honestly strive for an understanding of the mysteries of life. Or as a fundamentalist Christian might put, the lie that the devil sold me to win my soul and take me to a hellish eternity.

The bible is a collection of works with many different human authors over a period of a few thousand years. Or again if you’re a fundamentalist Christian, the Bible is a collection of works all authored by God but entrusted to a number of different humans over a period of a few thousand years. Regardless of your position on this, the defining argument for fundamentalists is that "The Bible is the written word of a perfect God and therefore infallible". OK, so let's break this down. I don't even want to get into the debate here about the inconsistencies and contradictions in the Bible which are well documented. I will instead assume that if you are a biblical literalist, then you will have arguments against these supposed "contradictions", or you are blissfully ignorant of them and don't care, in which case I'm surprised you've read this far.

So from the definitions here, Firstly God is "perfect". This by definition, at least in my mind, means that God when communicating with say the angels or even just thinking aloud to himself, or creating the universe by his "word", would not use an imperfect human language to perform this act of divine communication. So God's native concept of "language" would be drastically different to anything any human actually speaks. OK then, but God wanted to communicate with us his created cognisant beings to explain his plan for our lives, and his elaborate rules for gaining salvation and entering into eternity in the nice place as opposed to the horrible place once our mortal bodies had stopped functioning. This would seem an important thing for such a God to want to do. He decides that the best way to do this is to reveal it to various people over a few thousand years in their native language. Right here is where I have my first problem. Translation is imperfect. It is notoriously easy to translate things incorrectly even between very similar human languages let alone between a perfect divine language and an imperfect human language. There is an old saying "The translator lies". So even in the original Ancient Hebrew, Ancient Aramaic and Ancient Greek that the collection of books we know as the bible was written in, there is one level of translation involved. To get to where we are today though, which is to say to a Bible written in our modern languages, involves at the least one more step of translation from one of these ancient languages which are no longer in normal every day use, and are now more for academic interest, into our modern language. The sheer amount of opportunity for inaccuracy in this step renders the task frightfully problematic. This is evidenced by the sheer number of different versions of the bible that are available in English alone, and the constant debates between different adherents of their favourite translations. At this point my belief that what we call the Bible has any claim to being "the written word of God" crumbled and shattered into pieces.

One might be tempted to argue that even though their are different translations the underlying doctrinal beliefs that come from the Bible still hold true. In response, I would simply point to the number of different denominations and factions in the Christian church today. If God really thought it was that vital to have a core set of doctrines then he's doing an extremely rotten job of communicating it to modern humans.

Life after Biblical Literalism

This was the start of the slippery slide that made me question everything I ever believed about Christianity. It made me realise that the Bible should not be given special treatment and be elevated above the critique given to any other historical document. It gave me the freedom to stop taking everything in the Bible as literal truth and to be able to understand it as a collection of books written in a particular period of time for very specific purposes that the people in those times had a far better chance of understanding than I ever will, with my language and my subjective understanding of reality. That's not to say that I don’t still find parts of the bible unbelievably inspiring and enlightening. The elegance of Jesus' words "let he who is without sin cast the first stone" is amazingly poignant and very applicable to our lives today. The problem is, the Bible is statically locked in time and place, and as a consequence there are a great many things for which it does not and cannot speak. Biblical literalists try desperately to extrapolate, but in my opinion just end up causing pain both for there own adherents and for others who they try to inflict their out-dated ideas onto.

One of the things that bothers me about my whole journey to this point, is why it took me so long. It wasn't until I was 25 that I really started to ask first-order questions about the faith I was raised with. The only conclusion I can come to is that the combination of the brainwashing working so well, and my own personal insecurities as a young adult caused me to want to keep believing it for so long.

Saturday, November 12, 2011

.Net Performance and Scalability Tuning Tip #2

The Database Bottleneck

Why is my database so flaming slow?

Have you heard this before? I have. I’ve also heard it as a good reason not to use relational databases. There are a number of reasons why a particular database could be slow, but lets take a step back and try to understand what a relational database is actually trying to do. Relational databases are designed to be a mathematically structured, transactionally consistent and optimized way to store and retrieve large amounts of “relational” data. If you are not storing relational data, then don’t use a relational database, it is not optimized for your usage scenario. Problem solved… well, perhaps not.

Let’s assume you are storing relational data, but the database still seems slow. Well, I could go in to detail about database tuning, but I’m not a DBA, and there are people out there who are far better at that than I. Also database tuning tends to get very vendor specific at its deepest levels. Besides, that’s not what I really want to focus on here. Maybe you could look at the performance of your Network, or the disks and other hardware your database server is running on, but again, I’m not network or hardware engineer, I’m a software engineer. So let’s fix it in software… or at least, to the best of our ability. I’m going to assume then that your database is well tuned and maintained by someone who is experienced in doing so, and knows all the different ways in your organization that this database is going to be used. Once we make these assumptions, then we can no longer say “the database is slow” I would go further and say that any modern database is actually very efficient for what it does. Still I have seen developers in this case trying to blame the database for the poor performance of the application.

Most often the database resides on another box entirely to the one you’re application is running on. This is where the problem begins. To actually get data from a database you have to create a database connection, that then opens a network port, which then negotiates a conversation over a protocol, asks the database server for some data. The database engine then does some processing to find your requested data or perform your operation, and finally returns the results as data packets over the network back to your application. Now that mightn’t seem like much, but compared to grabbing that same data from your local memory, or even a file on your local disk, we are talking many orders of magnitude difference. Even if you are using a database engine on your local machine, it will almost certainly be running in a separate process which means that although you don’t have a physical network to navigate, you still have inter-process communications protocols to deal with, and this can still be a few orders of magnitude difference to getting the data directly from memory.

The most important thing about developing against a database is to have respect for the overhead required when querying the database. Based on this respect we can establish a few basic rules about how to interact with a database.

1. Don’t keep asking the database for the same thing over and over again. If it’s something you need regularly cache it. Caching comes with its own problems, the biggest of them being when to invalidate it, but the one thing you can be certain of is that it won’t be slow.

2. Try to ask the database as few times as possible for information during any one operation. Ideally it would be great if you could simply have a single SQL statement that gets everything you need to perform the desired operation, and it is worth investing a few (thousand) CPU cycles trying to figure out exactly what data you are going to need for a particular operation before hitting the database. It is also usually better to return a little more data than you might need, than to find you have to hit the database again because you forgot something. Of course this can be taken too far, so be sensible here, you shouldn’t be returning your entire customer and products tables just to process a single order.

3. Learn SQL, or your vendor specific implementation of it at least. Understand how to form the queries that get you the data you want, and take advantage of the well tuned database (assuming it’s well tuned). If you are using a tool that generates SQL for you, like LINQ2SQL or Entity Framework, understand how it generates SQL. These tools tend to produce sub-optimal SQL because they are trying to solve very generic problems, however, they can be coaxed into doing things more efficiently if you understand how they work.

If you still think the database is slow, I would suggest doing some profiling of your application. SQL Server comes with a profiler that is more than adequate. Other database engines have their own profiling tools. The Redgate ANTS performance profiler I talked about in my previous post has a SQL profiler (warning it doesn’t work with SQL Express), that is integrated with the code performance analyser. Also just a hint if you’re using ANTS performance profiler, you need to look at the “Wall Clock” setting when you are analysing a piece of code that has a lot of SQL interaction as the CPU clock will give you a false impression.

There are many other possible performance issues that can occur with databases, but this is all I really want to talk about at this stage. I have seen many projects in my time that have suffered because the 3 rules above have not been followed.

Monday, September 12, 2011

Travelling Again

I am currently travelling overseas, partly for work partly for pleasure… actually mostly for pleasure. As with other times, I find that travelling to foreign countries creates a lot to reflect on, and consequently write about. My first stop will be California. I am currently in a hotel in Los Angeles, on Monday I will be driving to Anaheim where I will be attending the Microsoft Build Windows conference, then I am planning to head to Santa Barbra for a bit of R & R. After that I will be visiting Buenos Aires. I will use this my personal blog as essentially a travel diary, but my geek blog as a means of recording my thoughts on the conference.

Sunday, September 11, 2011

Microsoft doesn’t care about feedback from Australians

I’m attending the Build Windows 2011 Conference in Anaheim this coming week, and as part of this Microsoft sent me out an email (presumably to all attendees, or at least all who indicated that they were a developer) inviting them to share feedback in special limited numbers discussion groups. Firstly though, they require you to fill out a quick survey. So I click on the provided link, and I’m presented with the following question.

image

OK, fine, I’m not too proud to be lumped in with “the rest of the world” team, so I click “Somewhere Else”, then “Next”. I am then presented with this

Thank you for your interest in our survey. Those are all the questions we have today. We hope that you will consider participating in future surveys.

OK, so basically, if you are not from any of these 8 countries, then your opinion is completely irrelevant to Microsoft. Regardless of the fact that you are the technical architect for Windows/PC version of a product that has over 400,000 user licenses world wide. Regardless of the fact that you are fostering a product that is responsible for direct license sales of SQL Server 2008 R2. The message is loud and clear, if you’re not in Microsoft’s G8, then your opinion is worthless. I wonder if our Mac Technical Architect (when he is eventually appointed) will get the same kind of treatment from Apple?

Really, the least they could have done is to let me continue with the survey, and then disregard the information anyway, I mean it really doesn’t hurt to store a few extra bytes.