Some Assembly Required

Musculosenioritis

mike-woodring — Wed, 13 Aug 2008 15:40:21 GMT

[muhs-kyuh-loh-seen-yawr-eye-tis]

-noun Pathology

soreness and weakness of the muscles, generally following athletic physical exertions reminiscent of the sufferer’s younger days; typically accompanied by the onset of stiffness and reduced ambulatory mobility during the 24-72 hours following the cessation of athletic activities.

[origin: August 4-13, 2008; musculo- + senior + -itis]

I finally have a diagnosis. Over the past few years, I’ve periodically suffered from a near-debilitating condition that results in great difficulty walking as well as moving from a sitting to a standing position; especially following prolonged periods of programming involving only the movement of my fingers and eyes. Until today, nobody could tell me what it was I was suffering from.

This condition generally sets in after I compete in athletic events (soccer, in my case), where I run, jump, and otherwise exert myself as if I were still a strong, fit athlete younger than 24 years of age. Alas, it has become increasingly apparent to me that this is no longer the case. In my case, the breakthrough diagnosis came last night after the adult, co-ed soccer team I’m a member of completed our run at this year’s trophy. We competed in 4 matches over a 4 day period last week, had the weekend off to ice and medicate our aches and pains, and then kicked off this week by competing in 2 more matches over the past 2 days.

As I sat on the sidelines after our loss last night, physically depleted and sapped of every last scrap of energy within me, I wondered what had brought me to this low point in my life. What malady was it that I could feel setting in even as I sat there limp with fatigue? What was it that caused such foreboding of what was to come this morning – a condition that I knew would worsen over the next few days as I sat here at my computer, typing lambda expressions and disassembling the contents of memory pointed to be the EIP register? What was it that could cause such physical pain merely attempting to rise from my chair and walk upstairs to refill my coffee cup?

And then it came to me - a word that so perfectly describes the condition suffered by myself and so many other over-the-hill athletes around the world: musculosenioritis.

Yes, I finally have a name for my tormentor. Like so many other afflictions, there is no known cure. Some would point to studies that indicate the condition can be ameliorated by abstaining from athletic competition clearly suited to younger, more physically fit personnel. But abstinance is no cure – it’s avoidance. And I will always choose to face my challenges head on; without quitting.

And so I will take what comfort I can just knowing that my affliction now bears a name, and that I am not alone in my suffering.

Fun with Anonymous Delegates & Lambda Expressions

mike-woodring — Tue, 01 Jul 2008 02:48:40 GMT

During the course of teaching a customized offering of .NET Framework Fundamentals last week, I was showing a historical progression of delegate syntax options. As I’ve said before, I’ve found that explaining something new to an audience in terms of something the audience already understands, is remarkably efficient. Most of my audience already had some experience with the .NET delegate syntax on the 1.x platform, so the case study I was using to illustrate the progression of delegate syntax options was a call to Array.FindAll<T> that looked something like this (the people variable is of type Person[], where Person is a simple class not shown here):

// .NET 1.x named method + explicit construction:
//
Person[] subset = Array.FindAll(people, new Predicate<Person>(LastNameStartsWithD));

bool LastNameStartsWithD(Person p)
{
    return p.LastName.StartsWith("D");
}

...which can also be rewritten as...

// .NET 2.x named method + implicit construction:
//
Person[] subset = Array.FindAll(people, LastNameStartsWithD);

bool LastNameStartsWithD(Person p)
{
    return p.LastName.StartsWith("D");
}

...which can also be rewritten as...

// .NET 2.x anonymous method + implicit construction:
//
Person[] subset =
    Array.FindAll(people, delegate(Person p) { return p.LastName.StartsWith("D"); });

...which can also be rewritten as...

// .NET 3.x lambda expression:
//
Person[] subset = Array.FindAll(people, p => p.LastName.StartsWith("D"));

This progression through the historical options for using delegates, coupled with reviewing the IL produced by the C# compiler for each of the above examples to see how they’re equivalent, makes it pretty easy to calibrate one’s eyeball in order to read & understand lambda expressions. But once we reached the lambda expression syntax, the class started to ask a few questions that all started out “So does that mean you can (insert crazy use of lambda expressions here)?”; to which the answer was invariably “yes”.

So during a layover on my trip home, and in the spirit of previous obfuscated programming contests, I decided to see how many ways I could (ab)use anonymous methods and lambda expressions to rewrite the canonical “Hello, world.” program using Visual Studio 2008.

Here’s what I came up with:

using System;
using System.Runtime.Remoting.Messaging;
using System.Threading;

class Program
{
  static void Main()
  {
    Console.WriteLine("Hello, world.");

    ((ThreadStart)delegate { Console.WriteLine("Hello, world."); })();
        
    ((ParameterizedThreadStart)delegate(object s) { Console.WriteLine(s); })(
        "Hello, world."
    );
        
    ((ParameterizedThreadStart)(s => Console.WriteLine(s)))("Hello, world.");
        
    (new Thread(s => Console.WriteLine(s))).Start("Hello, world.");
        
    ((ParameterizedThreadStart)(s => Console.WriteLine(s))).BeginInvoke(
      "Hello, world.",
      ar => ((ParameterizedThreadStart)((AsyncResult)ar).AsyncDelegate).EndInvoke(ar)
      ,null
    ).AsyncWaitHandle.WaitOne();
        
    ((ParameterizedThreadStart)(
      s => {
        Thread.CurrentThread.IsBackground = false;
        Console.Write(s);
      })).BeginInvoke(
        "Hello,",
        ar => {
          Console.WriteLine(ar.AsyncState);
          ((ParameterizedThreadStart)((AsyncResult)ar).AsyncDelegate).EndInvoke(ar);
        },
        " world."
      ).AsyncWaitHandle.WaitOne();
  }
}

I’m not yet sure what this says about my mental health :-)

?LINQ is the new Active?

mike-woodring — Tue, 11 Sep 2007 14:49:00 GMT

Note: Don and I go way back, and this post was just intended as good natured ribbing. But in light of the fact that at least one other colleague could not tell that from the post, I figured I should point out explicitly that (a) I like and respect Don very much, (b) I wrote the following only because I thought the “sound“ of the PLINQ acronym was worth a chuckle, (c) I was recently a bystander to an in-depth discussion on LINQ's history that Don participated in [ergo, I used his name below], and (d) my sense of humor is apparently slighly malformed. So inspite of having had another colleague of Don's and mine give my post a once-over, I may have once again demonstrated how difficult it is to convey elbow-in-the-ribs joshing via a written medium. As for the actual technical merits of LINQ or PLINQ, it should be noted that I have no opinion whatsoever, let alone a negative opinion. I've been preoccupied elsewhere and haven't had the time to develop an opinion yet. So I offer my sincere apologies to Don if he too couldn't tell I was kidding, and hereby point out for future reference to everyone else that I'm just kidding...

After hearing for months about DLINQ and XLINQ, I suppose I shouldn't have been surprised this morning to crack open the October 2007 issue of MSDN mag and find an article about PLINQ (Parallel LINQ). It reminds me of the ActiveX days, when it seemed like every technology or product that came out was somehow "Active": ActiveSync, ActiveGrid, ActiveCrash, ActiveThis, ActiveThat, ... Today, it's ThisLINQ, ThatLINQ, and TheOtherLINQ.

So after reading the PLINQ article, I was a quite disappointed to have it confirmed that Microsoft has once again over promised and under delivered. It reminds me of the difference between the COM+ that Mary Kirkland wrote about in 1993 compared to what COM+ actually looked like when it shipped in 2000; or (more infamously) the original hype surrounding the wonders of XML Web Services.

What I'm talking about, of course, is the fact that PLINQ represents only the barest of features that Microsoft originally promised those of us at last year's SDR in the Grand Caymans, where their plans for a parallel query execution framework were first described. At the SDR, they went on and on about how TIDDLY WINQS (Threaded, Interactive Dynamic Data Layer Yields for Windows(r) In Networked Query Systems) would provide fully automated, language integrated, support for GRID computing by parallelizing queries across .NET-based systems connected in a P2P environment.

At the time, Don Box, with his little rum-based, umbrella-laden drink, flowered shirt, and carefully orchestrated disheveled appearance ("Look at me, I'm too brilliant to bothering getting a haircut or combing my hair!") waxed grandiose about how TIDDLY WINQS would automate support for massive scale computing by breaking complex query expressions into discrete sub-expressions that would be farmed out over the network to other cooperating "players" (PC nodes), each of which would compute the result for their assigned sub-expression, and then later "chip" (submit) the result of their computation into the appropriate TIDDLY WINQS "cup" (SQL Server 2005 Cluster). Only when every "player's" sub-query results were harvested, would the TIDDLY WINQS framework synthesize the final result and return the aggregate (”winning”) result to your application. And since TIDDLY WINQS (via a pluggable "connectivity provider") would automatically handle the loss or reacquisition of network connectivity, Don went on and on about the virtues of using TIDDLY WINQS in ClickOnce apps using WPF and WCF, but running in a PCE (partially connected environment).

So it came as a great disappointment to me this morning to realize that after all of the TIDDLY WINQS hype, Microsoft instead delivered PLINQ, which appears to be aptly named after the soft, dull thudding sound of the impact this unimpressive technology will like have on next-generation .NET applications.

HxD Hex Editor

mike-woodring — Thu, 05 Apr 2007 01:19:00 GMT

Over the years, I've used a variety of hex editors. In recent years, I've been lazy and just used the binary editor built into Visual Studio. But I recently discovered HxD, a free hex editor that I've really enjoyed working with. It loads and runs as fast as notepad, has a nice tabbed UI for working on multiple documents, and supports a ton of really nice features; among them:

search by text string, hex, int, or float value
search & replace
insert mode
disk editor
RAM editor (view/edit the memory of a running process)
nice printing support
view bytes as groups of 1, 4, 8, or 16 bytes

So if you're ever in the market for a free, fast, nicely done hex editor, be sure to check out HxD. It's been a nice little addition to my toolbox.

WinDbg Bug Update

mike-woodring — Mon, 22 Jan 2007 17:44:00 GMT

Following up on this post, Microsoft has confirmed this to be a bug in version 6.6.7.5 of windbg. The good news is that the fix is scheduled to appear in the next release. The bad news is that, while there's no official date for that release, ballpark estimates from Microsoft indicate that it won't be for another 3 months or so.

WinDbg Bug: Disassembling JITed Code

mike-woodring — Mon, 15 Jan 2007 21:31:00 GMT

As a follow up to this post, it would seem that the disassembler in version 6.6.7.5 of WinDbg appears to be broken when it comes to disassembling normal JITed code. I haven't noticed any other issues, but this one is a killer for me, so I've reverted to using version 6.6.3.5, which does not suffer from disassembler bug.

Disassembling native or pre-JITed code also appears to work. The bug seems to be isolated to the disassembly of normal JITed code. So if you're doing much work in windbg with managed code that involves looking at the disassembled output of managed methods, you might consider reverting to version 6.6.3.5. If you need the new/updated features/fixes of version 6.6.7.5, be advised that the disassembly you see for normal JITed methods may be buggy.

Note that you'll see the same buggy output using the windbg u command as well as the sos extension's u command. That's because sos.u doesn't implement its own disassembler - it just defers to the active debugger's disassembler; providing CLR-aware annotations to the output. As a result, if you use the !sos.u command within the context of the Visual Studio 2005 debugger (which does not appear to have this disassembler bug), you'll see valid output.

Ironically, if you look at the what's new page for version 6.6.7.5 of windbg, you'll find this line item:

Updated disassembler with new instruction support.

Thanks, but no thanks.

My test case is a simple C# console application that looks like so:

class Program {
  static void Main() {
    System.Console.WriteLine(GetText());
  }
  static string GetText() {
    return ("Hello, buggy windbg!");
  }
}

Using the !sos.bpmd extension command to break when Program.Main is called shows the following disassembly in version 6.6.3.5 of windbg (using !sos.u so that we can see the CLR-aware annotations):

0:000> !u eip
Normal JIT generated code
Program.Main()
Begin 01200070, size 23
01200070 56               push    esi
01200071 833dc82da20000   cmp     dword ptr [00a22dc8],0x0
01200078 7405             jz      0120007f
0120007a e87f22e978       call    mscorwks!JIT_DbgIsJustMyCode (7a0922fe)
0120007f 90               nop
01200080 ff152c30a200     call    dword ptr [00a2302c] (Program.GetText(), mdToken: 06000002)
01200086 8bf0             mov     esi,eax
01200088 8bce             mov     ecx,esi
0120008a e855901b78       call    mscorlib_ni+0x2f90e4 (793b90e4) (System.Console.WriteLine(System.String), mdToken: 06000764)
0120008f 90               nop
01200090 90               nop
01200091 5e               pop     esi
01200092 c3               ret

In particular, note that the disassembled code shown above for instruction address 0120008a. The raw instruction sequence at that address is e855901b78. As I described in my previous post, the 1st byte (e8) is the Intel 'call' instruction, and the following 4 bytes (55901b78) indicate the target of the call instruction, computed relative to the instruction pointer, and not counting the 5 byte instruction itself. So we can manually compute the target of the call instruction in windbg like so:

0:000> ? 0120008a + 5 + poi(0120008a + 1)
Evaluate expression: 2033946852 = 793b90e4

The above calculation shows that the target of the call instruction is address 793b90e4, which matches windbg's output:

...
0120008a e855901b78       call    mscorlib_ni+0x2f90e4 (793b90e4) (System.Console.WriteLine(System.String), mdToken: 06000764)
...

Repeating the same experiment with version 6.6.7.5 of windbg (the buggy version) shows the following:

0:000> !u eip
Normal JIT generated code
Program.Main()
Begin 01200070, size 23
01200070 56              push    esi
01200071 833dc82da20000  cmp     dword ptr ds:[0A22DC8h],0
01200078 7405            je      0120007f
0120007a e87f22e978      call    mscorwks!JIT_DbgIsJustMyCode (7a0922fe)
0120007f 90              nop
01200080 ff152c30a200    call    dword ptr ds:[0A2302Ch]
01200086 8bf0            mov     esi,eax
01200088 8bce            mov     ecx,esi
0120008a e855901b78      call    MSVCR80!_NULL_IMPORT_DESCRIPTOR+0x2cdd (781b9055) (MSVCR80!_NULL_IMPORT_DESCRIPTOR)
0120008f 90              nop
01200090 90              nop
01200091 5e              pop     esi
01200092 c3              ret

There are a couple of things wrong with this output:

the sos.u command doesn't realize that the 2nd call instruction goes to Program.GetText
it incorrectly reports that the 3rd call instruction will land somewhere in MSVCR80.DLL (at address 781b9055).

Not likely.

Performing our same call decoding math for instruction 0120008a (the supposed call to address 781b9055 in MSVCR80.DLL) yields the following:

0:000> ? 0120008a + 5 + poi(0120008a + 1)
Evaluate expression: 2033946852 = 793b90e4

Note that our manual decoding of the call instruction yields a different target address: 793b90e4; as opposed to 781b9055 as shown by the !u command. If you disassemble that address, we'll see where execution will actually transfer:

0:000> !u eip
preJIT generated code
System.Console.WriteLine(System.String)
Begin 793b90e4, size 35
793b90e4 57              push    edi
793b90e5 56              push    esi
793b90e6 8bf9            mov     edi,ecx
793b90e8 ba86000000      mov     edx,86h
793b90ed b901000000      mov     ecx,1
793b90f2 e8d1a9ab00      call    mscorwks!JIT_Writeable_Thunks_Buf+0x198 (79e73ac8) (JitHelp: CORINFO_HELP_GETSHARED_GCSTATIC_BASE)
793b90f7 8bf0            mov     esi,eax
793b90f9 837e6800        cmp     dword ptr [esi+68h],0
793b90fd 750a            jne     mscorlib_ni+0x2f9109 (793b9109)
793b90ff b901000000      mov     ecx,1
793b9104 e80347f9ff      call    mscorlib_ni+0x28d80c (7934d80c) (System.Console.InitializeStdOutError(Boolean), mdToken: 0600070f)
793b9109 8b4e68          mov     ecx,dword ptr [esi+68h]
793b910c 8bd7            mov     edx,edi
793b910e 8b01            mov     eax,dword ptr [ecx]
793b9110 ff90d8000000    call    dword ptr [eax+0D8h]
793b9116 5e              pop     esi
793b9117 5f              pop     edi
793b9118 c3              ret

And, in fact, that's exactly what the C# code we're looking at does at that address - call System.Console.WriteLine.

So be advised - the disassembly included in WinDbg version 6.6.7.5 is broken when it comes to decoding normal JITed code.

WinDbg Disassembler Confusion

mike-woodring — Fri, 22 Dec 2006 02:28:00 GMT

[Update: I forgot to mention the observation that, in the buggy installation of windbg, the target address that windbg shows for the call instruction in question appears to be the raw operand value for the call instruction -- without treating that value as relative to the current EIP. Update appears below.]

I've had a sneaking suspicion lately that some of the disassembled code I've been looking at in WinDbg for managed code that's been JITed has been a little off kilter. But it only seems off when it comes to the disassembly for JITed code. The disassembly for normal native code looks fine. And if I step through the JITed code, it actually executes just fine. It's just that the disassembler seems to be out of whack.

So this afternoon I sat down to see if I could clear things up (prove/disprove my suspicion). Unfortunately for me, I believe I've proved my suspicion (the windbg disassembler is out of whack), but I haven't been able to correct it. So I thought I'd post my results here on the off chance that someone out there's had a similar experience and can explain what's going on.

I happened to be poking around at the managed exception handling mechanism, so the code I'm looking at looks like this:

class Program {
  ...
  static void ThrowException( Exception ex ) {
    throw ex;
  }
}

In WinDbg, I used the SOS !bpmd command to set a breakpoint on the ThrowException method. When the breakpoint hits, here's what the disassembly looks like on a machine (XP Pro SP2) where windbg seems to be working just fine:

Breakpoint 1 hit
...
0:000> u eip
01200420 56              push esi
01200421 8bf1            mov esi,ecx
01200423 833dc82da20000  cmp dword ptr ds:[0A22DC8h],0
0120042a 7405            je 01200431
0120042c e8cd1ee978      call mscorwks!JIT_DbgIsJustMyCode (7a0922fe)
01200431 90              nop
01200432 8bce            mov ecx,esi
01200434 e89340e978      call mscorwks!JIT_Throw (7a0944cc)
01200439 cc              int 3

The line of code in question is the one at address 01200434 (the call to mscorwks!JIT_Throw). WinDbg is kind enough to disassemble the instruction and its operand (e89340e978), and show us that the target address for that call instruction is 7a0944cc. But if you wanted to do it yourself (which will be come germane in a moment), WinDbg figures that out by parsing the instruction at address 001200434 like so…

The first byte (e8) is the x86 'call' instruction, which in this case is followed by a 32bit operand indicating the target address of the call relative to the current instruction. That value (9340e978) is displayed as a sequence of bytes above, so to see what the actual relative offset as a 32bit value, we can do this:

0:000> dd eip+1 l1
01200435  78e94093

So the target address is at 'offset' 78e94093 relative to the current instruction pointer (after the opcode and its operand have been consumed, which is 5 bytes). So we can do the math ourselves in windbg like so:

0:000> ? eip + 5 + poi(eip+1)
Evaluate expression: 2047427788 = 7a0944cc

Note that this results in the same target address (7a0944cc) that windbg had shown us was the call to mscorwks!JIT_Throw. Just for completeness, we can confirm this to be the case like so:

0:000> u 7a0944cc
mscorwks!JIT_Throw:
7a0944cc 6894000000      push    94h
7a0944d1 b8d497317a      mov     eax,offset mscorwks!GetManagedNameForTypeInfo+0x1b355 (7a3197d4)
7a0944d6 e848f3ddff      call    mscorwks!_EH_prolog3_catch (79e73823)
7a0944db 8bf9            mov     edi,ecx
7a0944dd e8761bf0ff      call    mscorwks!ResetCurrentContext (79f96058)
7a0944e2 8d4d84          lea     ecx,[ebp-7Ch]
7a0944e5 e81a09deff      call    mscorwks!LazyMachStateCaptureState (79e74e04)
7a0944ea 85c0            test    eax,eax

So that's the reference case on the machine where windbg seems to be in working order. Repeating the same exercise on the buggy installation of windbg (on the same app, using the same version of windbg, XP, and the CLR), here's what I see when the same breakpoint hits:

Breakpoint 1 hit
...
0:000> u eip
00d20420 56              push    esi
00d20421 8bf1            mov     esi,ecx
00d20423 833d102f910000  cmp     dword ptr ds:[912F10h],0
00d2042a 7405            je      00d20431
*** WARNING: Unable to verify checksum for C:\WINDOWS\assembly\NativeImages_v2.0.50727_32\mscorlib\4918a15d4f2a914781651294adafd2ce\mscorlib.ni.dll
00d2042c e8cd1e3779      call    mscorlib_ni+0x2b1ecd (79371ecd)
00d20431 90              nop
00d20432 8bce            mov     ecx,esi
00d20434 e893403779      call    mscorlib_ni+0x2b4093 (79374093)
00d20439 cc              int     3

Right off the bat, it's apparent that windbg is trying to locate symbols for mscorlib for some reason, even though this code doesn't call mscorlib. Also, windbg isn't able to convert what it believes is the target address in mscorlib into anything useful in terms of symbols. Furthermore, disassembling what windbg indicates is the target function being called (79374093) yields what I refer to in technical jargon as gobbletygook:

0:000> u 79374093
mscorlib_ni+0x2b4093:
79374093 2900           sub dword ptr [eax],eax
79374095 b801000000     mov eax,1
7937409a 8986880a0000   mov dword ptr [esi+0A88h],eax
793740a0 83be880a000001 cmp dword ptr [esi+0A88h],1
793740a7 0f94c0         sete al
793740aa 0fb6c0         movzx eax,al
793740ad 5e             pop esi
793740ae 5f             pop edi

Note, BTW, that the address windbg's disassembler computes (79374093) is actually the raw operand value in the call instruction:

0:000> dd eip+1 l1
00d20435 79374093

So it would seem that windbg, on this installation, isn't computing the target of the call relative to the current instruction pointer - it's just displaying the operand for the call instruction as-is.

To top it all off, if I single step (using windbg's 't' command) into the target method, I correctly 'land' at mscorwks!JIT_Throw:

0:000> u eip
00d20434 e893403779 call mscorlib_ni+0x2b4093 (79374093)
00d20439 cc int 3
0:000> t
eax=00913148 ebx=0012f4ac ecx=0128a084 edx=00000000 esi=0128a084 edi=00000000
eip=7a0944cc esp=0012f448 ebp=0012f478 iopl=0 nv up ei pl zr na pe nc
cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000246
mscorwks!JIT_Throw:
7a0944cc 6894000000 push 94h

When I first saw this, I thought my symbol cache was corrupted or otherwise out of date. But doing .reload /f /o (even after manually deleting my local symbol file cache) didn't change the output. Since the Pentium doesn't seem to have any issues actually making the method call, I decided to manually decode the call instruction; comparing the same code on a working installation of windbg and this buggy one. So with the instruction pointer sitting at the address of the call instruction in question, I performed the same computation of the call target as shown above:

0:000> ? eip + 5 + poi(eip+1)
Evaluate expression: 2047427788 = 7a0944cc

THAT looks a lot better, since it corresponds to where the Pentium actually landed when the 't' command was used above. But just to round off the operation, here's confirmation of where that call instruction is going to land:

0:000> u 7a0944cc
mscorwks!JIT_Throw:

7a0944cc 6894000000      push    94h
7a0944d1 b8d497317a      mov     eax,offset mscorwks!GetManagedNameForTypeInfo+0x1b355 (7a3197d4)
7a0944d6 e848f3ddff      call    mscorwks!_EH_prolog3_catch (79e73823)
7a0944db 8bf9            mov     edi,ecx
7a0944dd e8761bf0ff      call    mscorwks!ResetCurrentContext (79f96058)
7a0944e2 8d4d84          lea     ecx,[ebp-7Ch]
7a0944e5 e81a09deff      call    mscorwks!LazyMachStateCaptureState (79e74e04)
7a0944ea 85c0            test    eax,eax

This little exercise would seem to prove that the windbg disassembler is screwed up on one of my machines. The really frustrating thing (other than the time I've lost on this issue) is that I can't seem to repair windbg. I've blown away my symbol cache as mentioned earlier, removed the Debugging Tools for Windows, and downloaded (again, just to be sure) and then reinstalled DTW from scratch - all with no change in behavior. I've also compared the help/about screenshots of windbg's about box on the good/bad installations, and verified that they're supposedly identical.

The only difference between the two machines I've been comparing is that one (the working one) is a regular WinXP machine, while the other (the buggy one) is the same setup running in Virtual PC 2004. But the OS, CLR, and tools on the host and guest OS are (as near as I can tell) identical.

Basically, I'm stumped.

Suggestions, anyone?

Google Code Search

mike-woodring — Thu, 05 Oct 2006 20:39:00 GMT

I just read a notice about Google's search facility specifically targeting searches of source code: Google Code Search. Have only given it the once over, but it looks interesting (primarily because it supports regular expressions and source language specific filtering, as well as other filtering capabilities).

Obfuscated CLR/C# 2.0

mike-woodring — Mon, 25 Sep 2006 21:35:00 GMT

Towards the end of a course I was teaching last week, I gave my students an obfuscated programming challenge designed to test their understanding of several areas of both the CLR and C# that we'd been discussing throughout the course. In typical fashion, students had to predict the output of the program without the aid of any compilers - and they had to explain their answer to receive the prize.

However, I kept the code small enough to fit on a single projector screen, which prevented me from including a few things I had originally planned to test them on.

For your mental noodling enjoyment, the code below is the enhanced version of that challenge - unrestricted by screen space dimensions. Enjoy! (And remember - you can only use your eyes, brain and, if you like, a pencil and some paper.)

// Obfuscated CLR/C# 2.0 programming challenge.  Predict the output of this program without
// using any compiler to build & run the code.  Explain your answer.
//
// Mike Woodring
// http://www.bearcanyon.com
// Bear Canyon Consulting LLC
//
using System;
using IAR = System.IAsyncResult;
using IE1 = System.Collections.IEnumerable;
using IE2 = System.Collections.IEnumerator;
using MRE = System.Threading.ManualResetEvent;
using PTS = System.Threading.ParameterizedThreadStart;

class P
{
    static void Main()
    {
        MRE fe = new MRE(false);
        MRE pe = fe;
        foreach( char c in new MSG() )
        {
            MRE ce = new MRE(false);
            PTS m = delegate(object s) { P p = (P)s; p.W.WaitOne(); Console.Write(p.C); p.G.Set(); };
            m.BeginInvoke(new P(c, pe, ce), delegate(IAR ar) { ((PTS)ar.AsyncState).EndInvoke(ar); }, m);
            pe = ce;
        }
        fe.Set();
        pe.WaitOne();
        Console.WriteLine();
    }

    public P(char c, MRE w, MRE g) { C = c; W = w; G = g; }

    char C;
    MRE W;
    MRE G;

    class MSG : IE1
    {
        public IE2 GetEnumerator()
        {
            int i = 0;
            foreach (char ch in new NTE<SM.B>()) yield return (i++ == 0 ? char.ToUpper(ch) : ch);
            yield return (',');
            yield return (' ');
            foreach (char ch in new NTE<SM.A>()) yield return (ch);
            yield return ('!');
        }
    }

    class NTE<PT> : IE1
    {
        public IE2 GetEnumerator()
        {
            foreach (Type t in typeof(PT).GetNestedTypes()) { yield return t.Name[t.Name.Length - 1]; }
        }
    }

    class SM
    {
        internal struct A
        {
            public struct cow { }
            public class too { }
            public interface car { }
            public enum will { }
            public struct pad { }
        }

        internal class B
        {
            public class wish { }
            public struct see { }
            public interface hill { }
            public enum ball { }
            public class boo { }
        }
    }
}

Quickly Attaching a Debugger to the Right Instance of w3wp.exe

mike-woodring — Wed, 16 Aug 2006 17:37:00 GMT

On certain machines, I'm repeatedly using remote desktop to access a Windows 2003 server so that I can debug something running in the context of ASP.NET (web app, web service, etc). Since various apps and services on these machines are segregated in different app pools, the first thing I have to do when I run something like VS.NET and bring up the attach-to-process dialog is figure out which running instance of w3wp.exe is the one I need to attach to. Sometimes I get lucky and pick the right instance on the first try. Other times I miss and have to try the others one at a time. While this isn't something that takes a tremendous amount of time, the repetitive nature of this drives me nuts.

So I finally sat down and wrote a little script to automate this annoying procedure for me. The result is a javascript file that I can just run with the name of the app pool I'm interested in debugging on the command line, like so:

c:\>apdebug AcmeAppPool
Microsoft (R) Windows Script Host Version 5.6
Copyright (C) Microsoft Corporation 1996-2001. All rights reserved.
Active app pools...
  AcmeAppPool (2860)
  MSSharePointAppPool (3268)
  DefaultAppPool (3652)
Launching debugger for 'AcmeAppPool' (2860)...

The script enumerates the running instances of w3wp.exe, displaying the app pool they host and their corresponding process ID, then launches the JIT debugging dialog if the app pool specified on the command line was active. Since I'm very often interested in the same app pool on a given machine, a desktop shortcut to that script with the right app pool name on the command line gets me to the right spot with just a double click.

A more detailed description, with sample output and a screenshot, along with links to the script source you can view and/or download for your own use can be found here.

The Code Monkey Song

mike-woodring — Tue, 09 May 2006 17:21:00 GMT

Other than the part about Tab, this song about code monkeys is brilliantly accurate (and therefore funny :-).

Via Jonathan Coulton's Thing a Week 29.

Captured Arguments for Anonymous Methods are not [Serializable]

mike-woodring — Fri, 24 Mar 2006 22:12:00 GMT

Here's one for the "Huh…I didn't know that…although now that I do, it's probably not that important"-department.

When you let the C# 2.0 compiler capture variables for use within the body of an anonymous method you're passing to someone, the class that the compiler synthesizes for holding & passing the captured arguments to the target method is not marked [Serializable]. For example, consider the following program:

using System;

class Program
{
    static void Main()
    {
        AppDomain ad2 = AppDomain.CreateDomain("Math Domain");

ad2.DoCallBack(CallDoMath); // Okay.
ad2.DoCallBack(delegate { DoMath(2, 2); }); // Okay.

        int x = 3;
        ad2.DoCallBack(delegate { DoMath(x, x); }); // SerializationException.
    }

    static void CallDoMath()
    {
        DoMath(1, 1);
    }

    static void DoMath(int a, int b)
    {
        Console.WriteLine(
            "[{0}] {1} + {2} = {3}",
            AppDomain.CurrentDomain.FriendlyName,
            a, b, a + b
        );
    }
}

At (1), a request is made to invoke the CallDoMath method in a different AppDomain than the one Main is executing in. AppDomain.DoCallback expects a reference to a CrossAppDomainDelegate. Since the signature of CallDoMath matches the signature of CrossAppDomainDelegate, the compiler is happy to expand ad2.DoCallBack(CallDoMath) into ad2.DoCallback(new CrossAppDomainCallback(CallDoMath)) for me. The result is that CallDoMath executes in the context of the AppDomain named "Math Domain".

At (2), an anonymous delegate is used to call DoMath(2, 2) "directly". In typical anonymous delegate fashion, this approach just saved me from having to write a stub like CallDoMath that conforms to the signature of CrossAppDomainDelegate. Instead, I've let the compiler write that little method for me. The results are the same as for (1).

But at (3), I've reference a local variable named 'x' within the body of the anonymous method that will call DoMath(x, x). That variable is a local variable located on the callstack for the Main method. In this situation, the compiler conspires to 'capture' that variable into an object on the heap, the reference to which is passed to the compiler-synthesized anonymous method. So even though it looks like I'm referencing the variable 'x' directly from within the anonymous method, I'm really dereferencing a field of an object. This line of code results in a SerializationException because (a) it's a cross-AppDomain call, which requires that all parameters be either [Serializable] or extend MarshalByRefObject and (b) the compiler-synthesized class that holds the captured variables is neither.

In retrospect, this is easy enough to see using ILDASM or Reflector. But even though I've looked at the compiler-synthesized class for captured arguments before, it's not something I ever paid attention to until I bumped into it a few minutes ago tinkering with some multi-AppDomain code.

At any rate, there's a bit of trivia for the archives…

TinyURL Updated

mike-woodring — Wed, 22 Feb 2006 18:10:00 GMT

If you're using my TinyURL utility that I originally described here, you'll need to grab the latest bits in order for the utility to continue working. It stopped working sometime this month (thanks to Medardas for bringing it to my attention). I had guessed it was due to some small change in the HTML response from tinyurl.com that was confounding my regular expression search for the result. But it turned out to be due to an HTTP 417 error ("Expectation failed") that was being returned by the server in response to the POST my utility makes.

In particular, the POST request my app was making included an Expect=100-continue header that needed to be removed. Unfortunately, I don't have any packet traces of my original development effort that would confirm who's side of the HTTP conversation changed. Did a recent Windows update cause a change in behavior of the .NET Framework's implementation of HttpWebRequest (and/or supporting code)? Or did tinyurl.com's server-side software undergo an update? I have no idea (not that it matters, other than satisfying my curiosity).

At any rate, once I diagnosed the issue with some help from Ethereal, the fix was an easy one. I just needed to turn off the 100-continue expectation on my side of the conversation before sending my request:

ServicePointManager.Expect100Continue = false;

With that fix in place, everything is back in working order. So be sure to grab the updated bits before your next CTRL-SHIFT-T.

Generic Type Inference & Constructors

mike-woodring — Mon, 12 Dec 2005 18:18:00 GMT

Reflecting over types has always been a pretty straight forward exercise. But reflecting over generic types:

public class Foo<T> {
...
}

and/or types that have generic methods:

public class MathUtils {
public static T Max<T>(T x, T y) {
...
}
}

adds an extra element of fun to the exercise. For example, given this class definition:

public class MathUtils {
public static int Max(int x, int y) {
return (x > y ? x : y);
}
public static T Max<T>(T x, T y) where T : IComparable {
return (x.CompareTo(y) > 0 ? x : y);
}
}

how would you use reflection to locate only the generic version of MathUtils.Max that takes a single generic type argument (T), has two parameters whose type is the same as the generic type argument, and that requires that type to implement the IComparable interface? For the archives, here's one possible solution:

MethodInfo maxMethod =
Array.Find(
    typeof(MathUtils).GetMethods(),
    delegate(MethodInfo mi)
    {
      if (mi.IsGenericMethod && (mi.Name == "Max"))
      {
        Type[] genericArgs = mi.GetGenericArguments();
        ParameterInfo[] parameters = mi.GetParameters();

        return(
          (genericArgs.Length == 1) &&
          (parameters.Length == 2) &&
          (parameters[0].ParameterType == parameters[1].ParameterType) &&
          (parameters[0].ParameterType == genericArgs[0]) &&
          typeof(IComparable).IsAssignableFrom(genericArgs[0])
        );
      }

return (false);
}
);

Depending on what you're doing, there are any number of criteria that you might use to locate one or all methods that satisfy a certain requirement. The Array.Find and Array.FindAll methods are a pretty good starting place. They both take an generic array and a Predicate delegate that's applied to each entry of the array. Then depending on whether you called Find or FindAll, the first or all matching entries in the array are returned.

What I bumped into today was a realization caused by my desire to write a simple helper class that basically facilitated the use of Array.Find and Array.FindAll in the particular area of reflecting over types. Basically - just a set of overloaded Find and FindAll methods that catered to locating methods that matched a given name, a given predicate, or both; and in flavors that assumed a default set of BindingFlags or allowed you to specify the BindingFlags. Here's how my definition of the helper class started out:

public class MethodFinder {
Type _targetType;

public MethodFinder(Type targetType) { // Constructor #1
_targetType = targetType;
}

public MethodFinder(object obj) // Constructor #2
: this(obj.GetType())
{
}

public MethodInfo Find(Predicate<MethodInfo> methodMatcher, BindingFlags bindingFlags) {
return Array.Find(_targetType.GetMethods(bindingFlags), methodMatcher);
}

// Various other Find overloads not shown...

public MethodInfo[] FindAll(Predicate<MethodInfo> methodMatcher, BindingFlags bindingFlags) {
return Array.FindAll(_targetType.GetMethods(bindingFlags), methodMatcher);
}

// Various other FindAll overloads not shown...
}

There's not a ton of gain by doing it this way in terms of typing reduction. The gain for me was in the set of overloads for MethodFinder.Find and FindAll that provided name matching default implementations, allowed for the use of default or caller-supplied binding flags, and the ability to instantiate a single MethodFinder to operate on a given type and then proceed to perform multiple searches over that type. In fact, what I found interesting as I worked through this little exercise had nothing at all to do with reflection. I just included the code above for its sample code value for anyone googling around for examples of using reflection with generic types or methods :-)

Here's the issue. As defined above, the MethodFinder class provides two constructors: one that takes a Type parameter indicating the type to be operated on, and another that takes an object reference whose type is to be operated on. That's fine as far as they go, and it's the kind of design that's commonly found on the 1.x .NET platform. But if you have in your hands a value, as opposed to an object reference, then you'd want to avoid doing this:

Rectangle rc = GetRectangleFromSomewhere(); // Rectangle is a structure.
MethodFinder finder = new MethodFinder(rc); // Invokes constructor #2.
MethodInfo intersect = finder.Find("Intersect");

Because the above code used constructor #2 of the MethodFinder class, and because Rectangle is a structure, while the argument to constructor #2 is a reference, a boxing operation ensues. At this point, I thought - "No problem - this is a great place for generics!" What I wanted was to just add a 3rd overload for my constructor that would accept a generic type argument:

public class MethodFinder {
Type _targetType;

public MethodFinder(Type targetType) { // Constructor #1
_targetType = targetType;
}

public MethodFinder(object obj) // Constructor #2
: this(obj.GetType())
{
}

public MethodFinder<T>(T obj) // Constructor #3 (illegal)
: this(obj.GetType())
{
}
}

If I did this (so went my thinking), then this line of code:

MethodFinder finder = new MethodFinder(rc);

would no longer incur the overhead of boxing, because the compiler's type inference abilities would cause it to prefer constructor #3, which would work out to the equivalent of the MethodFinder class having provided an overload for its constructor that looks like so:

public class MethodFinder {
public MethodFinder(Rectangle obj)
: this(obj.GetType())
{
}
}

...which wouldn't incur any boxing overhead. The problem is that C#'s type inference capabilities (xref the C# 2.0 spec, section 20.6.4, Inference of type arguments) only apply to generic methods other than the constructor. So what I tried above was illegal. The only way to get a generic type parameter into the parameter list for a constructor is to define the enclosing type itself as a generic type:

public class MethodFinder<T> {
...
public MethodFinder(T obj) // Constructor #3 (legal)
: this(obj.GetType())
{
}
}

Or better yet:

public class MethodFinder<T> {
...
public MethodFinder() // Constructor #3 (legal)
: this(typeof(T))
{
}
}

Either of the above approaches would eliminate the boxing issue with value types like the Rectangle structure used above. The problem with this approach, however, is that I've now turned the entire MethodFinder class into a generic type. So while I've made it possible to use the MethodFinder class efficiently on value types, I've made the MethodFinder class more of a pain to use in non-generic scenarios. This is because any use of MethodFinder now has to "pass" the generic type argument:

SomeClass obj = GetSomeClassInstanceFromSomewhere();
MethodFinder<SomeClass> finder1 = new MethodFinder<SomeClass>(); // Generic finder.
MethodFinder<SomeClass> finder2 = new MethodFinder<SomeClass>(obj); // Finder for instance.
MethodFinder<SomeClass> finder3 = new MethodFinder<SomeClass>(typeof(SomeClass)); // Finder for type.

Because I've made MethodFinder a generic type, and because the C# compiler doesn't infer type for generic type arguments, I've lost the simpler usage model I used to have:

SomeClass obj = GetSomeClassInstanceFromSomewhere();
MethodFinder finder1 = new MethodFinder(obj); // Finder for instance.
MethodFinder finder2 = new MethodFinder(typeof(SomeClass)); // Finder for type.

Argh! That's when it dawned on me that there was a very simple solution to providing a class design for MethodFinder that catered to both the generic & non-generic scenarios. Instead of trying to define a single class that provided all of the constructor forms I was after, I just needed to do a bit of refactoring. A non-generic base class provides the actual functionality, while a generic derived class supports the usage model for the value type scenario:

public class MethodFinder {
Type _targetType;

public MethodFinder(Type targetType) { // Constructor #1
_targetType = targetType;
}

public MethodFinder(object obj) // Constructor #2
: this(obj.GetType())
{
}
}

public class MethodFinder<T> : MethodFinder {
public MethodFinder() // Constructor #3
: base(typeof(T))
{
}
}

With this solution, both the value type and reference type scenarios are supported. And because of the inheritance relationship between MethodFinder and MethodFinder, you can use a MethodFinder anywhere a MethodFinder is expected:

MethodFinder finder;
finder = new MethodFinder<Rectangle>();
finder = new MethodFinder(typeof(SomeClass));

SomeClass obj = GetSomeClassInstanceFromSomewhere();
finder = new MethodFinder(obj);

All that said, I should note for the record that there's also another way to solve the problem. The primary purpose of this post was to discuss the approach of mixing a generic & non-generic types in an inheritance relationship. But the other way to deal with the fact that the C# compiler doesn't support type inference of generic arguments for constructors is to stick with one non-generic class definition that supports non-generic constructors, and just add generic factory methods as needed that takes the place of a generic constructor:

public class MethodFinder {
Type _targetType;

public MethodFinder(Type targetType) { // Constructor #1
_targetType = targetType;
}

public MethodFinder(object obj) // Constructor #2
: this(obj.GetType())
{
}

public static MethodFinder Create<T>( T obj ) { // "Constructor" #3
return new MethodFinder(typeof(T));
}
}

This works as well. And some people might find it preferrable to defining two classes. But others probably don't like the idea of a class that supports normal constructor as well as factory-based construction. I'll leave that decision for the reader. At any rate, it's food for thought...

Constraints on constraints

mike-woodring — Tue, 06 Dec 2005 16:09:00 GMT

I was fiddling around the other day with generics, generic collections, and anonymous methods, looking to see if they could be combined to provide a compact, elegant utility class and/or method that would make facilitate firing & forgetting an event asynchronously using the thread pool.

The closest I could come was a solution that would let me do this to fire the event:

Utils.FireEvent(
SomeEvent,
delegate(SomeEventHandler handler)
{
handler.BeginInvoke(argshere, delegate(IAsyncResult ar) { handler.EndInvoke(ar); }, null);
}
);

But I don't find that to be particularly readable. And it's certainly not more compact or elegant than the straight forward approach:

SomeEventHandler handlers = SomeEvent;

if( handlers != null ) {
foreach( SomeEventHandler handler in handlers.GetInvocationList() ) {
handler.BeginInvoke(argshere, delegate(IAsyncResult ar) { handler.EndInvoke(ar); }, null);
}
}

So in the end, I chucked the idea and stuck with the straight forward approach to using the thread pool to fire and forget an event. But along the way, I ran into a restriction (constraint) on the way type parameter constraints work that I had not bumped into before.

Here's how my Utils.FireEvent method started out:

public class Utils {
public static void FireEvent<TDelegate>( TDelegate eventHandler, Action<TDelegate> itemHandler ) where TDelegate : Delegate
{
    if( eventHandler == null ) {
      return; // Nothing to do.
    }

    TDelegate[] handlers =
      Array.ConvertAll(                // 1
        eventHandler.GetInvocationList(),                   // 2
        delegate( Delegate d ) { return (TDelegate)d; }     // 3
      );

    Array.ForEach(handlers, itemHandler);                   // 4
}
}

My intention was to use Array.ConvertAll (line 1) to convert the Delegate[] array returned by Delegate.GetInvocationList() (line 2) to a more specifically typed array of TDelegate references (handlers) that could then be passed to Array.ForEach (line 4). The conversion from Delegate to TDelegate would be performed by the anonymous method passed to ConvertAll on line 3. My thinking had been that since I constrainted TDelegate to a derivative of System.Delegate (where TDelegate : Delegate), the compiler would ensure that callers didn't try to pass some non-delegate argument to this method. The typecast on line 3 (from Delegate to TDelegate) could certainly still fail, but the caller would just be getting what they deserved :-)

But the above approach doesn't compile. The compiler issues the following error for the constraint declaration:

Constraint cannot be special class 'System.Delegate'.

Sure enough, the C# 2.0 specification has this to say (20.7, Constraints):

A class-type constraint must satisfy the following rules:

The type must be a class type.
The type must not be sealed.
The type must not be one of the following types: System.Array, System.Delegate, System.Enum, or System.ValueType.
The type must not be object. Because all types derive from object, such a constraint would have no effect if it were permitted.
At most one constraint for a given type parameter can be a class type.

I can't find any further discussion of why a type parameter cannot be constrained to deriving directly or indirectly from System.Delegate. (For the record, System.MulticastDelegate isn't allowed either.) So the reasoning behind that constraint on constraints is still unknown to me. If anyone has some background on this, I'd appreciate your comments.

My next thought was to back off a notch with the constraint and just use a reference type constraint:

public class Utils {
public static void FireEvent<TDelegate>( TDelegate eventHandler, Action<TDelegate> itemHandler ) where TDelegate : class
{
...
}
}

This change eliminated the error about my illegal constraint, but still results in an error on line 3:

delegate( Delegate d ) { return (TDelegate)d; } // 3

Cannot convert type 'System.Delegate' to 'TDelegate'.

Unlike the first error I encountered, which informed me there was something special about Delegate that prevented the use I tried, this error gives me a little less to go on. My expectation had been that since I'd constrained TDelegate to be a reference type of some sort, that the typecast operation I was going to attempt on line 3 might or might not succeed - just like any other typecast. If d was in fact type compatible with TDelegate, the cast would work. Otherwise, the caller would get InvalidCastException. That seems reasonable with me.

As before, I went looking through the C# 2.0 spec to see if there was an explanation. The closest I could come was this sentence in section 20.7.4, Conversions involving type parameters:

The above rules do not permit a direct explicit conversion from an unconstrained type parameter to a non-interface type, which might be surprising.

The spec goes on to describe a scenario where trying to use an unconstrained type parameter in this fashion doesn't make sense. And I buy that. But I don't have an unconstrained type parameter. I've constrained TDelegate to being a reference type of one sort or another. So I'm still not clear on why the typecast syntax of line 3 should be disallowed.

Odder still is the fact that, although the typecast conversion above results in a compiler error, the following variation works just fine:

delegate( Delegate d ) { return(d as TDelegate); } // 3

Other than how a failure to perform the requested conversion from System.Delegate to TDelegate is conveyed to me and my caller (InvalidCastException versus a null reference), both variations are attempting the same type conversion. So it seems to me that either both forms should be allowed, or both forms disallowed. Allowing one, but not the other, doesn't make sense to me.

So if someone could shed some light on either or both of these two restrictions on type parameters and their constraints, I'd appreciate your comments.