The PATH to Enlightenment

In my day job, I've been relocated to an office nearer where I live, so I have a 20 minute commute each way instead of a 90 minute commute each way. I started at my new location on Monday, and started off by working on one of my AutoMod simulation models. I have a laptop, and this model was on its hard disk.

I quickly realized that my simulation was not loading, but appeared to have hung whilst reading in a kinematic movement system. Now AutoMod (I'm using version 10.0 for reasons that I may elaborate on in a future blog entry) does not have good tools for diagnosing this type of problem, so I had to try everything I could think of. It goes without saying that there was nothing in either am2out.dat or am2err.dat - or any of AutoMod's other output files - that would indicate what the problem was.

I tried importing the simulation, restoring it from an archive instead of the more usual binary format. Same problem.

I tried an older version of the same simulation that I knew was working just fine on Friday - at the other office. I wasn't sure whether I had modified, and re-loaded, the current archive/binary version of the model and I wanted to be sure that the problem was unrelated to any changes I had made. Nope.

I then edited the archive's model.amo file (after making a backup copy!) to remove all the archives, then added them back in one-by-one - doing an import after each addition - until I found which system was causing the problem. The model was able to successfully load a Power & Free movement system, a Path Mover movement system and an AS/RS movement system before it failed whilst reading a Kinematic movement system. (In fact, quite a few other Kinematic movement systems failed as well).

So I had my culprit! I then edited the offending archive definition for the Kinematic system in question to see if I could see anything obviously wrong with it. Nope - it looked fine.

The format of a Kinematic archive file is fairly simple (compared to some other movement systems), but one potential sticking point could be the cell file definition for the kinematic vehicles within the file. This is defined by a ROBOTDEF entry that initiates the cell file definition (but that is not itself a part of the cell file definition), and terminates with a redundant end statement. I cut and paste this definition into a cell file and tried to read it with ACE, the AutoSimulations Creation Editor - a tool that is provided with AutoMod. Guess what? ACE hung. So, it appeared that I had isolated the problem further to an embedded cell file definition.

To make sure that this was the problem, I edited the Kinematic archive system file (again after making a backup) to replace the cell file definition with something simpler - just an empty set with no children. I then imported the model again, to find that it loaded fine.

So the problem was the kinematic vehicle cell file definition.

So, I looked closer at the cell definition to see what might be causing the problem. I have written a tool that parses cell files and reports any errors with within them - it does a much better job of doing this than ACE - so I used it on the cell file. No problems - the cell definition was fine!

One of the things that my cell file tool reported was that the cell included a reference to a VRML (virtual reality markup language) file. Could this be the source of the problem? I suspected it might be, because the cell file was otherwise very simple.

So. To see if there was a problem with the VRML file, I started ACE and tried to read the VRML file directly. ACE duly hung. I then modified the cell file to remove the VRML file reference, and reloaded it with ACE. It was read in just fine!

So the problem was now further distilled down to reading in a VRML file.

To validate the format of the VRML file, I resorted to using a browser plug-in for navigating VRML files called CosmoPlayer. This excellent tool is no longer supported, but is available from NIST. It requires quite a bit of sleight-of-hand to get it to work with modern browsers, and does not work with recently-patched versions of Internet Explorer 6 and above, but I trust it's VRML parsing implicitly. When I browsed the file with Cosmo, it rendered the image fine and reported no problems. Curiouser and curiouser!

Now, this VRML file was generated by another tool that I have written and it basically looks something like this (simplified to protect the innocent and to improve clarity):

#VRML V2.0 utf8
EXTERNPROTO SomeObject [] "Prototypes.wrl#SomeObjectDef"
Group {
    children SomeObject {}

Not complicated, is it? Except that ACE was hanging reading it in. So what does this file contain? The file line is a pseudo-comment that specifies the VRML version and encoding. The second line states that SomeObject is a reference to an object prototype definition called SomeObjectDef in another VRML file called Prototypes.wrl. The last three lines simply include a copy of SomeObject as a child of the root of the VRML file's scene graph.

So. Could the problem be in the Prototypes.wrl file? This is a much more complicated, hand-written file that I created to store definitions of commonly used objects for my simulations. Now, at this point I should mention that the systems that AutoMod was able to read in just fine, before encountering the "bad" Kinematics system, included cell definitions that included VRML file images also. I tested ACE by reading in other VRML files, which made no reference to Prototypes.wrl, without any problem. So the problem had to lie with the prototype definition file. Right?

(I should point out that I hadn't made any modifications to any of these graphics files - or the tools that generate them - for months. So how could they suddenly stop working without somehow getting corrupted?)

Well. I could spend a lot more time on this, but, essentially, I couldn't find anything wrong with Prototypes.wrl! I had tried a number of VRML rendering tools and they all read in all of my VRML files without problem! I was starting to tear my hair out - what little I have left, after 20+ years in this industry...

So. I decided to call it a day and sleep on it.

By the next morning, it had dawned on me that the only difference between running the model on the previous Friday (when it worked) and running it again on the following Monday (when it didn't) was that I had changed offices. Could that be a part of the problem? How on Earth could that affect ACE's ability to read in a VRML file?

To address this, I booted my laptop without plugging in the network cable, and tried to load my original, unmodified simulation.

It worked just fine!

This blew my mind! So, I plugged in the network cable and tried again. It failed. What the...?

As a matter of expedience, I worked on my model with the network cable unplugged - after all, I was slipping behind schedule after all of the above - until I was ready to find out what the problem was.

At one point, I tried to copy some files off a network server that was located at my old office - and my laptop ground to a halt, taking over 30 minutes to download a small file to my laptop. "Hmmm", I wondered. Could this explain my problems with reading the VRML file?

Well, my simulation was not referencing any files on the network - after all, it read in just fine with the network cable unplugged. But it got me thinking: was AutoMod or ACE trying to reference anything on a server whilst reading the model?

It seemed like I was clutching at straws, but for some reason, I thought of my PATH - the environment variable that tells Windows were to look for executable files when I enter a command. I knew that my PATH included a directory on the very same server that I was having problems copying files off. So, I changed my PATH to remove this directory and re-ran ACE with the network cable plugged in - and it worked!

TGS provide the implementation of OpenInventor that is used by AutoMod to process VRML files. The version used in AutoMod 10.0 is OpenInventor 3.1. It appears that this implementation, or (possibly) AutoMod/ACE, looks at the PATH when encountering an EXTERNPROTO statement in a VRML file. If the server is not available, then there is no problem. But if it is, then I presume that it starts looking for a file there - and that's what causes the simulation to hang.

In fact, I used ACE to read in the original VRML file, with the network cable plugged in and with the original PATH, and - after a couple of hours - rendered the file correctly. So there were definitely no problems with the VRML file contents!

The moral of this story? Well, if AutoMod was an open source application, then I could have traced this problem way faster - and even (possibly) put a fix in for it. As it isn't open source, I had to go through all of the above to find what was - in the end - a very simple problem.

I'm guessing that everyone who has ever used AutoMod, or any other proprietary simulation package, has wasted hours of their time on issues like this.