The "RenderWall"
Those of you who read my blog regularly might recall my mention of the Graphstream Renderwall that we have in the office. Not having posted a Hardware article in a while, I thought it was time to talk about this marvelous piece of kit.
As listed on the ERDC MSRC SciViz Hardware webpage, the Graphstream is a rack of 13 Dual-proc 3Ghz Xeon machines with an NVidia 7800GTX in each. Now, the video cards we are using are dual-head capable. The first (primary) head of each node is hooked to a nice digital KVM switch, along with the 13th “Head” node. The second head of each system is hooked to a nice 1600×1200 native LCD monitor. Each node has both an Infiniband and Gigabit Ethernet interconnect. The motherboards are some very nice ones with Dual 16x PCI Express lanes. We originally did this to play with SLI, but decided to forgo that in favor of the improved performance of Infiniband. More on that in a bit..
So what does one do with a setup like this? Read on to find out. Well, we built a “Renderwall”. Essentially, 12 of the 13 displays were mounted to the wall in a 4 by 3 array. Each display runs at 1600×1200, meaning we have an effective 6400×3600 display on the wall. Each of the nodes runs Redhat Linux (the paid-for enterprise edition), and automatically boots up to X on the Primary display. MPI is configured on every node, so you can start an additional copy of X on each node to use the secondary head (:1.0 instead of :0.0), and write MPI aware applications using OpenGL to render across all the nodes. It’s a bit tricky, but it’s the kind of thing that you figure out once (like we already have) and then use it as a template for all your future apps.
That’s fine and dandy for experimental work, but what about stuff that’s already out there? Things like ParaView and Ensight? This is where the Infiniband really benefits us. From the same people who brought us Mesa (the Software OpenGL Emulator) there is a package called Chromium. Chromium, like mesa, operates as an OpenGL emulator. The main difference is that it has the extra knowledge and functionality to scale your display size and coordinates across all the nodes, and properly pass out the OpenGL commands to be executed in all their hardware accellerated glory on every node. This lets you take almost any OpenGL application and run it in 6400×3600 goodness! It’s not without it’s problems tho, the biggest one being the number of features supported. Things like Shaders and Fragment Programs are still in development, but for standard OpenGL1.2 or OpenGL1.0 stuff it works great. It’s a huge network hog, tho, as every GL command has to be broadcast to all the nodes. This is why something like Infiniband is needed, so we can get a nice 12Gigabit bandwidth.
And before you ask, Yes you can run Quake 3, using Chromium, in 6400×3600 resolution 
I mentioned this once before, but words just can’t adequately describe it. We haven’t gotten the opportunity to test it since the Infiniband was installed, but with just Gigabit ethernet it very playable around 30fps (rising to 80 and falling to 5 in spots).
The system truly shines when pushed to the limits. One thing that’s really powerful is running Paraview across the system. You can run a PVServer on each node, using the 8Gig of Ram & dual Xeon processors to parallel-process huge amounts of data. Then, you can use Chromium with the Client, to farm the display back out to the renderwall for some truly impressive results. This makes near-full utilization of every single node, and really shows what it’s capable of.
So one question that’s asked alot is “Why 6400×3600? Why not just hook a single PC at 1600×1200 to a projector, and fill an entire wall?” The answer: resolution. Sure you could make the display bigger with a projector, but if you set down the mouse & walk up to the screen you’ll see that it’s blocky. By increasing the actual resolution, you can get a similar physical size but keep the fine detail. For some of the massive datasets we use, these small details are the most important part! To see that small detail on a conventional display, you have to zoom way in on the dataset, which leads to problem of getting “lost” in the data. By retaining the resolution but increasing the physical space, you can add more surrounding data allowing context and spatial features to become apparent. Imagine trying to find a red M&M in a warehouse full of green M&M’s, while being forced to look through a microscope. What the renderwall does is the release this restraint, allowing the user to “take in” the entire warehouse at once.
Well, I hope you enjoyed reading this. We’re currently in the process of expanding to some larger displays, switching from 20″ LCD’s to 30″ LCD’s, or maybe even Projectors. There’s also a project named DMX, or Distributed Multiheaded X, that will let you run a normal Linux X session across the entire tiled display. This way you can run OpenOffice or web browsers across the entire display, good for presentations. We’ve done some experiments, but the seams between our LCD’s become way too annoying for any prolonged use of this.
No related posts.