Yeraze's Domain 3.0

At work we use Autodesk‘s 3dsMax as an integral part of our visualization process.  We use paraview and VTK to create geometry, typically as PLY files, and then import them into 3dsMax for high resolution renderings.  We have some nice scripts that will let us create a context scene, and then load each dataset in turn and submit a job to the render farm.  The results look amazing and it’s really taken our results to the next level.

Unfortunately, 3dsMax wasn’t really designed to do this and it’s really starting to show.  To manage a renderfarm (a large collection of nodes dedicated to nothing but rendering), they provide a piece of software called BackBurner.  Unfortunately, it’s pretty well tuned for controlling 2 or 3 jobs that each contain a few hundred frames.   Frankly, the performance really stinks when you have 2000-3000 jobs, each of 1 frame each.  Once a few jobs queue up we’ve seen submission times drop from 4-5 jobs per second, to 30-45seconds per job.  It’s pretty ridiculous, and once the number gets over 1000 the whole system just seems unstable and unreliable.  The manager seems to just “take a smoke break” as we’ve started calling it:  It simply stops responding and stops handing out new jobs for about 5 minutes, then suddenly catches up with everything that happened while it was out, never reporting an error.

What I’ld really like to do is submit a single job that could use MAXScript to dynamically load and unload data between frames.  This would greatly reduce both the amount of time to submit jobs (submit a single job instead of a job per frame or job per dataset), and reduce the load on BackBurner (Only has to manage 1 job instead of thousands).  Unfortunately, it seems there is no way to do this.  MAXScript has a preRender/postRender callback but they are only triggered at the beginning of the first frame and the end of the last frame, not between frames.   They do offer a preFrameRender/postFrameRender callback that offers that, but you’re not allowed to change the geometry.

The only other way I know to do this is using the existing preRender callback, but somehow getting backburner to restart 3dsMax between frames, instead of automatically advancing to the next one.  This I know will work, but I don’t know if it can be done.  Muster from Virtual Vertex can do this through control of the “Packet Size” option, so I may switch to that.  Honestly tho, I’ld really rather keep it in BackBurner, since it’s already installed & operational (mostly).

So I ask you, o great IntarWeb… Any ideas?
[tag:3dsmax][tag:autodesk][tag:maxscript][tag:backburner]

The DOD User’s Group Conference 2006 is coming up in June, this year in Denver, Colorado. At my boss’s advice, I submitted two papers this year:

1. ezViz – a Cross Platform, Open Source Visualization Tool . This is an update to the paper, by the same name, that I presented last year at UGC05, updated with the new features of ezViz and with some real case studies of real user data. This is a major thrust here at ERDC now, so it was the “more important” of the 2 papers.
2. Visualization of Time-Dependant, Quasi-Streamwise Vortex Tubes in a Bubble-Laden Turbulent Boundary Layer over a Flat Plate . Yeah, I didn’t come up with that name. This is a rehashing of some older data (that the lab won an award for), using new visualization methods presented by some folks at Viz05 (their paper: Opening the Can of Worms: An Exploration Tool for Vortical Flows ).  In their paper they were restricted to uniform rectilinear grids, so we expanded it to non-uniform grids and time-varying data.  It’s pretty neat stuff (very math intensive), but of lesser importance than the ezViz paper.

Well, the UGC06 website has finally been updated with an agenda, and now I’m a bit confused.  The “important” ezViz paper was relegated down to a Poster, while the “less important” vortex-tube paper was accepted as a presentation.  However, it was not lumped in with the rest of the Visualization presentations, it was placed as one of the last talks on the last day.  For those of you that don’t know, that’s basically a death knell for any conference presentation, everyone will leave early & be on their way home by then.

Apparently the number of submissions was an all-time record high this year, so they’ve got 5 rooms fully booked simultaneously with presentations, and a huge number of posters.  Word around the water cooler is that I’m not the only one a bit upset, as alot of major papers were turned into Posters.  Posters are 1) expensive, and 2) alot of hard work.  These guys don’t know much about making posters and would really rather just write the papers and presentations Powerpoint & LaTex (Yes, these guys really do use LaTex, how awesome is that?).  So maybe things will change before the conference comes around.  We’re already talking about switching out my 2 talks (Turn the vortex tubes into a poster, and the ezViz one into a presentation) since it’s the more important of the two, then maybe we can convince the committee to let us present in the “viz block”.
[tag:dod][tag:ugc][tag:conference][tag:paper][tag:visualization]

I’ve been searching for ways to make a few bucks on the side lately. Working with Tom is one way, but that’s alot of work. I always figured that there had to be an easier way and last week I think I finally found it.

Long ago I heard of a website called Scriptlance, a kinda “eBay” for programmers. People would post projects ranging from homework assignments to full web-portals, and then allow programmers to bid for the work. Theoretically this helps the people get the best deal for their money, and it gets alot of work out in the open. I never really thought it would work before, because you’ld have people undercutting for ridiculously low prices (college students, foreigners, etc). But, signup was free so I created an account and got to digging around. I have to admit I’m impressed.

I’ve finished 2 contracts through the website so far. Most of the work is pretty trivial stuff (for me anyway). But so far, the project that I’m most proud of is the ItsOnSirius.net project that I just wrapped up. It started as a small project to simply retrieve a text file of the current listing on Sirius Radio and generate a nice HTML table from it. A few hours with PHP later and it was done, easy. After that was running for a few days, he came back with another request for logging all the data into a MySQL database to allow for searching and playlists. That’s alot more ambitious, but a day or two of planning and 4 hours of coding later and it’s done and running. The guy is pleased with the results, and I found it pretty neat to do. I even got a chance to play with Plesk, a nice web-based system managment tool.

So I think I’m gonna keep doing the freelance programming thing for a while. It keeps me busy at nights and breaks up the monotony of games or movies every night. Plus, if I just take the minor jobs then I can easily take a day off. So if anyone out there reading this needs any work done, drop me a line… I just may take you up on it
[tag:freelance][tag:sirius][tag:programming]

I haven’t written about work in a while, so I thought I’ld post a quick update on what I’ve been doing lately.

We got another terabyte dataset in the office last week. Terabyte datasets used to be “whoa” but not so much anymore. With the new Cray XT3 coming online, we’re starting to see alot of them as simple “test” runs. We’re starting to wonder if this new linux cluster we’re getting is going to be enough horsepower when the “real” datasets start coming in. But anyway, I digress.

We got a big terabyte dataset in this week, 3 of them in fact. Three different test-cases, 400 timesteps in each run. The datasets themselves are pretty neat CFD simulations of flowing water around boats and such. Some of it seems to be repeats of the 5415 tests, probably for verification of the sim. We quickly got a pipeline up to convert the data into VTK format, and then using some of my software got isosurfaces extracted and rendered. Very kewl stuff.

Well I thought I would try a volume rendering. I had done that on an earlier dataset and the results were spectacular. I tried it again here and got nada. Just a big black square. Well, that’s not right. After some digging and experimenting, I figured out that Mesa support was missing in that particular piece of code. After some discussion and more digging, I managed to rewrite it to support Mesa and now the patch is awaiting approval from Kitware.

With that fixed, I happily went back to volume rendering. The initial test renders looked good, (Sorry, not cleared to post any pics here) but they really needed to be tweaked. Given that I have “occupancy” and “velocity” at each point, it seemed like a good idea to map color to velocity and opacity to “occupancy” (empty cells with no water will be transparent). I tried that, and more blank images. Ok, something else is wrong here. After alot more digging and discussion, I found a broken cut-n-paste and another Patch is awaiting approval from Kitware.

So that’s my last 2 weeks in a nutshell. It’s fun to work on stuff like this, although it is a bit annoying to make a change and then have to wait two hours for the rendering to run. It currently is a completely serial code, runs on just 1 processor, and requires 8G of ram to run. I’m running it on our old Onyx340 so it’s dog slow. Hopefully our new linux cluster will help alot here. We’re lookin at buying like 6-8 16-core opteron, 128G constellations, but we’ll just have to see what comes back from GSA.
[tag:vtk][tag:work][tag:kitware][tag:software][tag:visualization][tag:volume][tag:render]

It’s been a week without a post, so I’m sure you’re wondering what’s up. Well, it’s been a pretty rough week at the office, and I’ve had way too much on my mind lately to post. There’s some uncertainty here right now that might put me floating my resume again soon, but nothing official yet. I don’t want to start too many rumors, so I’ll just leave it at that.

Also this week we’ve had alot of vendors coming through. Folks from Dell, NVidia, and Verari all came through last week telling us about their plans for the future. Unfortunately, it’s all under NDA so I can’t talk about it. Suffice it to say there wasn’t too much talked about that you can’t already see on The Register, but tthere were a few little nuggets that we’re doing some more research on. One thing neat from NVidia was talk of their new rendering product “Gelato” and “Sorbetto” that you can get from http://film.nvidia.com . Essentially a GPGPU-accellerated version of tools like Mental Ray, it’s supposed to give you film-quality rendering at a fraction of the time of other packages by running on the NVidia Quadro GPU. It’s cross platform and works with Max & Maya, just like Mental Ray, so we’re curious how well it would work on our upcoming Linux Cluster. It would be nice to get rid of a separate Windows Cluster just for rendering.
[tag:job][tag:sgi][tag:gelato]
Also, today we got an ominous phone call from our SGI Sales Rep confirming what we had read a few weeks ago in The Register. Seems they’re dropping the Prism Line and going forward with Altix. Unfortunately, Altix is AGP-bus based where Prism is (was?) PCI-Express bus based, so the graphics capabilities will be sub-par. Hopefully they’re about to release a new version of the Altix with PCI-Express support, but until then it seems SGI is going to continue to lose market share to competitors. Seems they’re also undergoing a huge restructuring as they’ve phased out alot of older products (Neptune, Prism, etc) and fired pretty much their entire management (CEO, CTO, CFO, C*O basically).

So it’s been a pretty hectic week. Hopefully things will stabilize with the job soon and I can post an update. Until then, I wouldn’t expect too many updates as worrying about that has pretty much consumed 100% of my mental-cycles.

As most of you know, I received Honorable Mention in the IEEE Visualization 2005 Contest last year. The project was to visualize an older dataset from Bharat Soni of a windfield over the city of New Orleans, resulting from the 1996 Bright Field accident.

Last week down in sunny Orlando, Florida, the guys from CEI held CEIViz06, an annual conference focused mostly on the features and capability of Ensight. My boss, Paul Adams, was there presenting a paper on
“High-Quality Ray Tracing of Terascale Simulations” when he had an interesting discovery. The presentation right before his was by Professor Ahmed K. Noor, Director of the Center for Advanced Engineering Environments, Old Dominion University, NASA Langley Research Center. Entitled
“Visualization Technologies – Frontiers and Beyond”, it was a “pie in the sky” presentation of the future of vizualization and data analysis. Paul was really surprised to see him suddenly put up a slide showing my contest entry. Dr. Noor did give me proper credit, although he implied that it was done as an ERDC MSRC project which it wasn’t. Also, Paul told me that he said it was a visualization of the effects of Hurricane Katrina, which it obviously was not. My former master’s professor Robert Moorhead was there and corroborates his story, and was equally surprised since he knows Dr. Soni as well.

I was initially flattered, as imitation is the sincerest form of flattery. I was at first anxiously awaiting seeing his presentation when it was posted online, so I could post it here & use it in my “body of work”. It would be my first referenced work, so I was kinda excited. But the more I heard about it from Paul, Dr Moorhead, and others, the more worried I got. The ERDC MSRC has done work for Hurricane Katrina, and this wasn’t it. This wasn’t even close. Finally, the presentations were posted on CEI’s website, and this one was surprisingly not included. After a few failed attempts, I finally managed to contact Dr. Noor via his email and was somewhat surprised to hear that he essentially has no intentions to publish his presentation. Rather than having a Powerpoint (or equivalent) presentation, his is pretty much all Macromedia based and not very suited for distribution. He says that he mentioned during the presentation that the work was from before Katrina, but evidently that slight detail was lost amongst the crowd as several people think the visualization is from Katrina. He also seemed rather irate and offended at me bringing the entire thing up.

So at this point, I’m kinda at a loss. It’s basically beyond my control, it’s his word vs Paul and Dr Moorhead’s about what was said, and it seems I won’t get to see the presentation for myself. I kinda feel like I need to “defend” my work, but there’s really nothing I can do. I mean, he’s the Director of the Center for Advanced Engineering Environments at NASA Langley, and I’m basically a nobody in comparison. I wanted to find that this was all an honest mistake and easily remedied, but it seems that how it is now is how it’s going to stay. He did mention that he may be writing a paper for publication soon, using the information from this presentation, perhaps I’ll have the chance to correct any misconceptions then. I guess all I can do for now is just post here that I’m aware of the presentation, and that the work is not related to Hurricane Katrina.
[tag:katrina][tag:visualization][tag:conference][tag:cei]

It’s been one rollercoaster of a week in the office. Let me start at the beginning.

Monday
We have our usual morning meeting, until someone brings up the question of our workstations. For those of you who don’t know, we ordered brand new workstations with the end of our FY05 money. Brand new Dell‘s, dual-proc dual-core Xeon’s, 8Gig of ram, 4 RAID5+1 (Mirrored & striped). Very sweet machines. Ordered over the summer, and arrived in December. Needless to say, it’s not December anymore and we’re getting a bit antsy. We’ve placed the blame squarely on the desktop support group, since they have the PC’s and they’re required to set them up. Unfortunately, they’ve been inundated with other work related to recent security audits and such, and our machines have fallen in priority.

Talking about this stirred up a huge hornet’s nest, as the message makes it’s way through management and gets progressively distorted with each retelling.  The desktop group takes personal offense to the implication that they aren’t doing their job. Right as it’s almost smoothed over, someone else chimes in and it gets even worse with the (now legendary) statement “I just want my computer to WORK”.

Tuesday
My boss takes a day off, leaving me to give a demo to a tour to a group from the “249th”. Unfortunately, every secretary and PR person in the building will be attending a meeting at that time, so I’m left completely up to myself to figure all this out. The demo goes well, although a bit klunkly in my opinion. On a good note, the vortex tube code I was working on long ago, I have finally gotten to work nearly perfectly. I wound up having to reverse-engineer the math the original researcher was using to generate his grid, which involved some pretty bizarre trig and calculus. But it’s done, and nearly perfect.

Wednesday
Boss is back, things are cooling down from Monday. I spend the day working on the vortex tube code, and get the last few kinks ironed out. So now I get to start the arduous task of writing up a paper on it for the UGC and IEEE Viz06 . (Yes, you read those pages correctly.. FInished papers by March 21) I have to go to bed early tonight, because tomorrow I have an 8am meeting in the HPC2 in Starkville, which means I have to leave here at 5:30am. Right when we think the events of Monday are smoothed other, tho, round 2 starts at 3pm. Luckily it remains civil this time, but I doubt we’ve heard the last of it.

So tomorrow I’ll be revisiting the old stomping grounds in Starkville, and hopefully I can schedule a quick trip by Little Dooey’s while I’m there. Then on Friday, who knows what cards fate will deal. I doubt a new desktop is in my near future, and expect I’m going to be getting very friendly with Microsoft Word.
[tag:work][tag:support][tag:papers]

One of the machines we have here at the lab is an older Compaq SC45 named Emerald. Given that the machine has pretty much reached end-of-life (4 years for government supercomputers), and that Compaq has been acquired by HP, this machine is basically “orphaned”. But since it is currently operational, that means I have to have my code running on it. For better or worse.

For those of you who have never had the “priviledge” of working on a Compaq SC, they run something called Tru64 Unix. I think it’s part of a standard called OSF Distributed Computing Environment, and also goes by the name “HP-UX” sometimes. I’m not sure how many of my problems are related to the fact that it’s based on this and how much is from Security and Age related issues, but it’s been an absolute nightmare. At first it wasn’t so bad. The GNU 3.4.0 Compilers that are installed don’t work (IOT/Abort Trap everytime), and alot of the GNU utilities I like were missing (top, vim, dir), but I was eventually able to get everything setup.

One of the things that really complicates my application is the amount of dependencies it requires. To get it running requires a working installation of (in Order) Mesa, VTK, Xdmf, HDF5, and Xmdf. To make it even more complicated, some of those require CMake. You can imagine how difficult it is to keep all these changing codebases in working order, but then multiply that by the amount of work it takes to keep it running on our SGI Origin3k running Irix, Cray XT3 running a variant of Linux, and various other workstations running Redhat Enterprise Linux. It’s pretty difficult, made even more so by the fact that each of these systems has their own compilers & libraries that are optimized for the architecture at hand. None of them are compatible, and each of them have their own problems.

This week I’ve been fighting with Emerald and Compaq’s (now HP’s) Tru64 compilers. I had previously gotten all my code to compile just fine, so I thought this would be an easy task. In classic Murphy fashion, it was anything but. The problem was an interesting one as everything would compile and link perfectly, no errors at all. I’ld install it all, and finally compile my program and install it beautifully. Then when I try to run it:

           emerald0> ./main
     resolve_symbols: loader error: dlopen: libvtkVolumeRendering.so: symbol “__array_new2″ unresolved

This led me on a two-day long adventure of the internals of dynamically-linked shared libraries and how they work. Specifically, how they don’t work. After two days of learning all about ldd and nm, and how /usr/bin/cxx is actually a symlink to a driver script in /usr/ccs/lib/cmplrs/cxx/V7.1-006… I finally figured it all out. As shown above, the problem was this mysterious “__array_new2″ function, which isn’t called by any of the source code that I’ve compiled (nor in my code nor any of it’s dependencies). I figured it had to be a result of name mangling of some other function, probably C++’s “new” keyword. Try as I might, I wasn’t able to figure out where it was tho. With a tip from some other guys, I finally found it in the libcxx.so library in the compiler’s directory. But why couldn’t the program find it?

Well, I was somewhat surprised to find out that even those these are “dynamically” linked and that they will find libraries specified via the LD_LIBRARY_PATH environment variable, evidently an initial search path is hard-coded into the program during the link stage. So what was happening was the “libcxx.so” link requirement was being satisfied by another libcxx.so in another directory, which didn’t have this __array_new2 function. Finally, after manually adding this directory to my compile commands & recompiling, I got it all to work. Adding this directory to my LD_LIBRARY_PATH wasn’t enough, as it was always looking in this hard coded “/usr/shlib” location for it first, and finding it there.

So what’s the point of this lengthy post? I honestly don’t know, I guess just to whine & vent a bit. That one stupid library cost me (and a user who was waiting for me) two days of productive work. So far the only reasonable explanation is that someone upgraded the compiler but didn’t remove some of the old libraries, hence this old cxx library in an odd location. On a positive side I learned a few things:

1. Always make sure you’re linking against what you think you’re linking against.
2. Always compile & install into your home directory first, before pushing it out to public use. I screwed up a perfectly good working installation by installing too quick.
3. This also helped us fix another problem that’s been bugging us for a long time: how to make Static Libraries. We would always get hundreds of errors about libraries not having a table of contents. Come to find out that the ar and ranlib in /usr/local/bin don’t work with the Tru64 outputs, and you have to use the other ar & ranlib in /usr/bin.

So in short, it’s been a very frustrating couple of days. But at least I finally accomplished what I originally intended (upgrade my app on Emerald), and learned a few new things along the way. Guess you can’t ask for more…. Except for it not to take a week.
[tag:Tru64][tag:programming][tag:compaq][tag:unix]

If you check my Published Work page you’ll see it’s had a bit of a face-lift. I was goofing around on Google and noticed that some new patent information has come out that finally lists myself (and various other Z-Kat’ers) as the Inventors!

So now, if you go to the US Published Application Full-Text Database Search site and search for “Randall” and “Hand” as the Inventory name, viola!
Four patents, or patent applications at least. It’s better than nothing, I suppose

The Fall 2005 Resource is finally published on the new ERDC MSRC website (it’s been in print for almost 2 months now, just finally got it on the website). I’ve updated my Published Work page to include the two articles in here (the same two articles from the User’s Group a few months ago), so feel free to check em out. You can also look on the back cover for examples of the type of work I’ve been doing lately.