Sunday, December 25, 2005

Fun with casts

Chris and I were discussing casting this morning…in particular consider this:

class B1 {
int b1_var;
class B2 {
int b2_var;
class D : public B1, public B2 {
int d_var;

This is an example of multiple inheritence. Now for a quiz:

D * d_obj = new D;
void * v = d_obj;
B1 * b1 = (B1 *) v;
B2 * b2 = (B2 *) v;

Now the question is: which of these pointers is safe to use? The answer is: on the compiler I tested (CodeWarrior 8) and probably a number of others, b1 is safe, b2 is not. But I wouldn’t trust b1 either - the programming techique isn’t a good idea!

The problem is that the address of an object and its base sub-object can only be the same for one base of a class. You can think of the C++ compiler as doing something like this:

class D {
int b1_var;
int b2_var;
int d_var;

In other words, it stacks up the memory layout of the derived class with the bases inside it so that they are equivalent. But clearly B1 and B2 can’t both have that handy first memory slot in D. The compiler handles this for you by automatically changing the pointer value when you cast. For example:

B1 * b1 = d; // This is safe!
B2 * b2 = d;

This is perfectly legal - in CodeWarrior the first statement assigns the address of D to b1. The second statement adds a few bytes to D and assigns it to b2, such that b2 points to the B2 subobject within D.

The reason the cast from a void * fails is that when casting between a class and an untyped pointer, the compiler cannot correctly adjust the pointer for inheritence. The result is that b1 and b2 have the same actual value as d - no cast involving a void * will change the address.

Making things even more confusing, the cast operator in C++ does 3 different casts. C++ provides explicit named versions: const_cast, reinterpret_cast, static_cast (and dynamic_cast, but this behavior is never available via a C-style cast). I recommend using the C++ casts since they make it clear what the code intends to do and makes you the programmer think about which cast you really want. (C casts are static casts when possible, but reinterpret_casts otherwise…very dangerous!)

The moral of the story is: if you are going to cast to void *, be sure to cast back to the exact same type!

Wednesday, December 21, 2005

The dangers of vector

Overall the STL has been a big win in X-Plane, but as I’ve worked on performance and optimization over the last week, some of the biggest wins have been from losing the STL and going back to older, simpler C structures.

The STL vector is a dangerous thing - most of the time it’s fast, efficient, easy to use, and simple enough that the compiler optimizes right through it. But when you have to build one up incrementally the performance is terrible.

The STL list can build up incrementally, but then for small objects you spend a ton on overhead for node pointer storage, etc.

So two things have worked well:

1. Simply adding a “next” pointer to small structures to turn them into an intrinsically linked list.

2. Using a memory sub-allocator. Our sub-allocator grabs a large chunk of memory, doles out small pieces (without tracking allocations — a high water mark just grows and we grab a new big block when we need one), and then the big blocks can be nuked en masse when we’re done with the whole subsystem. (Thanks to one of our users who I won’t “out” for suggesting this and design!)

In X-Plane we have to build up a lot of structures whose final sizes are difficult to determine, so these two ideas work well together; the sub-allocator is very fast for small allocations (the common case can even be inlined), and the overhead for the next pointer is reasonably painless. This scheme saved 25 MB just by applying it to the physical terrain triangle mesh.

(Another advantage of the intrinsically linked list: no need for a copy constructor or operator=, which might be difficult or undesirable to implement for a very complex heavyweight object.)

A final thought: putting 2 GB of RAM in your devepment machine might seem like a good idea, but it can be dangerous. Before I got the above scheme working my previous attempts to reorganize the code using STL vectors and lists bloated memory usage by 200 MB, and the performance hit was zero — the dangers of a G5.

PS Thanks to Chris for letting me glom up his blog.

Installshield and services

Well this one cost me 7 hours. After figuring it out, I’m amazed it took me more than 10 minutes but it’s a sneaky one.

Installshield has a host of functions for windows service control. ServiceStartService, ServiceStopService, ServiceAddService, ServiceRemoveService are the more common ones. I was attempting to write an installer that would be doing an upgrade of a product that had a windows service. The task seemed simple…Stop the service, remove the service, uninstall the old product, install the new product, install the new service, start the new service? It looked pretty simple as a state diagram on a scrap piece of paper too.

The problem:
Eventhough the service stopped succesfully, it wouldn’t remove successfully. I kept getting a 1072 error which means “The stated service has been marked for deletion.” This prevented my new service from getting installed properly since they have the same name. So when does this service actually get deleted? The moment you close Installshield. This means that i’d have to basically quit the installer and then start it up again at a different state. Hacky!

The solution:
DO NOT USE ServiceStopService! Stop the service the old fashion way with “net stop myService”. Here’s a code snippet:

LaunchAppAndWait(WINDIR ^ "system32" ^ "net.exe", "stop myService", LAAW_OPTION_WAIT|LAAW_OPTION_HIDE);

Also, the same issue applies to ServiceRemoveService. You should use sc.exe instead which should be standard. It’s in Windows/System32.

LaunchAppAndWait(WINDIR ^ "system32" ^ "sc.exe", "delete myService", LAAW_OPTION_WAIT|LAAW_OPTION_HIDE);

The cause:
This is 100% speculation on my part as i have been unable to find any confirmation of my reasoning but here goes. ServiceStopService does work…the service is successfully stopped however the means through which it stops the service acquires a handle to the service that belongs to the Installshield application. Services reference count open handles to themselves so that they don’t get removed during usage. So when you stop the service, you open a handle that doesn’t get closed until you quit the application. By using “net stop” no handle is acquired. It’s an old fashion DOS command.

Sounds like Installshield needs a disclaimer on their function documentation for this.

Thursday, December 08, 2005

DCOM and ASP.NET permissions…A Microsoft Nightmare

Why anyone would choose a Microsoft IIS server over apache is beyond me but all personal preferences aside, permissions for DCOM objects is a nightmare when they need to access remote data.

I had a bug to solve which ultimately boiled down to a DCOM object calling IFileSourceFilter::Load. Seems simple enough…Load merely takes a file path or url, whichever you prefer (in this case it was a URL). It always failed however with 0×80070002 “The system cannot find the file specified.” After checking all of the obvious things (did the file really exist, was the path properly formed, was I using an unsupported internet protocol) I decided to take this section of code out of the project and stick it in an EXE by itself. Lo and behold it worked just fine! Hmm interesting, it works in an EXE but not in a DCOM.

So now you’re thinking, permissions! So i checked the permissions on the DCOM object and everyone listed had permissions to do just about anything. Then i thought i was missing a user. When the DCOM object is run, it’s spawned by IIS and the owner is “ASPNET.” So i added that user to the DCOM permissions and gave it full permissions. Still no luck.

Finally after adjusting just about every IIS, Windows and .NET Framework security permissions, I began a quest to google just about every possible combination of “ASP.NET permissions, IFileSourceFilter, DCOM, file not found” that i could think of. Eventually I came across this bit of text from Microsoft.

When using , which is the default setting, ASP.NET attempts to delegate the ASPNET local user account. As this account does not possess any network credentials, to the network it appears as the Windows anonymous account (NT AUTHORITY\ANONYMOUS LOGON

Well now we’re getting somewhere. So the webserver (IIS) spawns a process that has no access to the web. Ok then…A bit more searching revealed this.

So i attempted impersonation on the whole web application.

That did the trick. But who really wants their whole web-app running as a domain user? No one. More googling revealed a C# class that accomplishes impersonation just long enough to spawn the DCOM object and run it with decent permissions. What a nightmare!

New Tech Blog

Well a ton of people have a personal blog and a ton of people have technical blogs and some people try to combine the two into one. I don’t think the geeks want to read about my personal life and I don’t think my friends want to read about my geek scribble so i’ve decided to separate the two into separate blogs.

I’m doing this mainly so that i remember the things i’ve learned along my career because all too often i stumble on a problem and when i ask for help i get the usual “Hmm i’ve seen this before but i don’t remember how we fixed it” response. I’m going to do my best to document what i learn so that i can say “Hmm i’ve seen this before and i don’t remember how i fixed it but i’ll search my blog and get back to you.”