User functions in Janus

Back somewhere around v1.6 I started developing the pre-/post-breakout script feature, where a script is run contextually to coincide with the breakout of a render pass. This matured to the script system in Janus in 1.7. But it was back then when I was in the mental thoroughfare thinking of ways to get user-created scripts to work with Janus. Specifically, I wanted to call functions from within the render pass cmd line; these functions should be completely user-defined, and obviously, referred to at run-time. This meant that I needed Janus to dynamically call a script. The wall I encountered was that because Janus is sourced and compiled, it can’t source (e.g. include/insert) any other script once it is distributed. I thought the gate was closed at the time.

Recently, I’ve broken through the wall by going through a surprisingly obvious route. In a word, the solution was to execute a single pre-defined script (at least for the initial implementation of the feature) which the user modifies or adds user functions.

The first step was to re-use the structure of how subcommand variables are processed, since they share pretty much the same procedure.

The second bit was to implement the Queue Object Agent in LScript into the procedure. The Queue creates slots which serve to communicate to the user-defined functions the function call name and its argument(s). The function name and arguments are derived from parsing the cmd line. Once the function name is located, and arguments identified, this information is inserted into the slots of the Queue.

Then comes the third step, which was to create the base user function script that Janus will call once the Queue is populated. This base script calls upon the Queue that Janus has created, and starts querying the slots for information. It gets the function name and arguments, then checks if the function name exists; runs the function within and expects a return; the return value is passed on back to a reserved Queue slot, whereby Janus picks it up again and replaces the whole function call with the return value.

Neat.

It took about thirty minutes to prototype and two and half hours to implement. More tests required, but I was surprised that I got it working that night.

I suppose it also bears noting that the motive of this effort didn’t just come out from the blue. I received an email from a client very recently asking how he can fine-tune his output file paths. He wanted to be able to rearrange the output path so that his version number stuck at the end. Because filenames are just single strings, there is no inbuilt way of identifying which is the base name against the version number; the whole filename is effectively the base name. A parsing solution is needed. But because this is Janus, hard-coding is frowned upon. The solution must come in a form of being able to programmatically process the render pass parameters so that the user can do anything with it, and free to modify it in case their circumstances change.

B52 R4

Janus 1.7 B52 R4: released it, and with it a price adjustment to $99 (from the original $199). But the maintenance update marked only bugfixes, and nothing spectacular in of itself. I think the price adjustment is more of the headline.

It’s a bit difficult to keep on reasoning that Janus should maintain the price point it has had for five years. If inflation had anything to say it should have gone up. Of functionality, Janus has increased in bang: in 1.6 where the interface was really streamlined and new workflows were introduced to form a more sophisticated base for later development, and in 1.7 where the offline method was implemented. But Janus dwarfs the price point of its fellow LW plugins – though it has no functional off-the-shelf peer in LW-land – and I’ve received hints from more than a couple of people of the price being the main turn-off.

But some also complain that’s its too technical, and I think that no price is going to compensate for a system that doesn’t feel right for them. Fair enough.

The many faces of Janus

When I typed in that title I wasn’t intentionally alluding to Janus, the Roman god. But it’s cool nonetheless.

I’ve been developing Janus for seven or so years, and I’ve faced, time and time again, the difficulty in presenting the Janus in a way that satisfies me – because I think it’s a good concept, it makes it even more difficult.

In my previous post about Dynamic Mesh Combining, and my recent re-vamping PSAT rendering I’ve been reminded of how cool coding ventures (from my point of view) have been dwarfed up by the pressure to promote Janus as a mainstream product, which presents the difficulty of consolidating the features that make up Janus into something that most LW users can understand and appreciate. And thus, I laboured two weeks on writing and creating an overview in the hopes that I have explained in a little bit better this time.

But this didn’t change the fact that there are a lot of stuff in Janus that’s difficult to present. On one hand you have For Loops, and on another you have Dynamic Mesh Combining; there is PSAT rendering, plugin support, buffers support, the proxy system, search-and-replace functionality (in object replacement, and throughout the system), the script system, the offline method, etc. Then you have these strange little terms like ‘composite command types’, which, if you were a Janus user, would make more sense – though it is most probable that even Janus users don’t touch it.

It’s though I’ve put in so much functionality that I wanted to have because I didn’t want to run another script to do it for me. One may venture to consolidate these features into a superlative blurb that it has ‘everything’. It doesn’t have everything, but it has a whole lot that no one knows about.

For Loops in Janus

In the recent 1.7 release of Janus I’ve implemented For Loops, which is a way to get a looping construct in the Janus context. It’s rather interesting, and it all came because a prospective (Janus) client emailed me one day asking me if it was possible to procedurally tell Janus to replace objects in a specific directory. There was none at the time, and the solution to that was to either to create render passes dynamically via definition files, or use composite command types. He was happy enough for such a solution. But for my part I felt things could be done better: it was still not procedural insofar as the generation of the data strings was external to Janus: what I wanted to do was get something procedural so that extra scripting was not necessary to do these sort of things.

Enter For Loops, which are represented as subcommands. For Loops tell Janus to ‘recycle’ the render pass (similar to what composite command types already do). In For Loop ‘recycling’ the render pass is internally duplicated; this duplicate passes through a string filter and replaces variable names which are a specific subset for the given For Loop. This contextualises the data that is being accessed; if the loop is around a directory, for example, then you can’t use the variable to get file line information, etc. However, being a construct, they do share some data, such as the iterator; perhaps more can be added in the future.

The prospective client didn’t end up getting Janus, though, as it often happens. But the great thing about it is his question sparked a good idea in my brain. It may also have been through my exposure with Houdini and its Wedge ROP that I’ve implemented it the way I have. These emails from prospecting clients was the root of another less-known feature of Janus called Dynamic Mesh Combining, which combines two or more mesh items into one and it does so during breakout so that the master scene is not modified in any way. Dynamic Mesh Combining came because a person asked me how to prevent shadows from being cast by separate shadow catcher. The only solution to his specific problem was to combine all of his shadow catching objects into one and then turn the Self Shadow render flag off. Because combining meshes is a tedious task, why not implement it?

Janus 1.7

I’ve just released Janus 1.7 which is a major update since a long time.

This cycle started when Mikael Burman, a Technical Supervisor working at Ubisoft Massive looked for a feature where Janus could break out passes without loading the scene in. Well, as it turns out, more than seven years ago, I was wishing that very same feature. Back then, however, the engine wasn’t mature enough; there were too many things to address, and the feature itself was a daunting task which, I had thought then, required an extended development. This feature was called the ‘offline method’; offline denotes that the scene file is not loaded into memory, but that the file will be parsed directly by Janus without pre-processing. I don’t know if this is Janus’s claim to fame (because Janus is not really that popular), but I  don’t know of any other render layer offering that can do that.

Had I attempted to do this back when I wanted it to, I very much think it would have taken me at least three times longer to do, and the coding wouldn’t probably be as clean that I could build a lot of future development on top of it. I credit the relatively slow and deliberate updates that led up to 1.6 with the robust base that I built the offline method on.

Janus’s breakout engine has two processing phases. The first phase is scene pre-processing, which prepares all objects for scene parsing; in this phase the items are ‘tagged’ with their own particular settings based on final evaluation of that pass. Key Layout functions are employed here which help in exporting shader-based overrides, and other peculiar tools such as mesh combine (combining separate meshes into one object during breakout).

The second phase is scene parsing, which opens the scene file and starts parsing the scene out, line-by-line, looking for keywords and tags, and dumping it to another scene – the breakout scene – with the data modified.

In the offline method, the second phase was generally left untouched, except perhaps for some additions to new tags that were related to the offline method that needed to be considered. It was actually the first phase that was particular different. In fact, the engine’s first phase is, for the most part, bypassed when the offline method is being used. Some of the code was still used, of course, because I couldn’t possibly think of changing the main ‘trunk’ procedures. I changed some of the called functions to imitate what was already going on in the ‘online’ method.

But this didn’t change the very base limitation of not being able to deal with shader-based overrides and dynamic mesh combination. Of course, these are actually possible, but it entails a lot more work. While I have some functions designed to write shader-based overrides directly into .lwos, my suite is not comprehensive enough for what is required. And dynamic mesh combination is a bit further away from my reach because I don’t have any code pertaining to handling .lwo mesh data.

I’m not sure if it’s worth the time, as well.

Lastly, there are still some things that can be addressed in the offline method. One disclaimed limitation is the fact that once you create your groupings in Janus, you cannot change them when the scene is unloaded, and the user is working in the offline method. This is because the groupings have been tagged into the items, and this used the parenting information from Layout to get those items tagged correctly. Perhaps there is a way to re-assemble the hierarchy internally when the the file is loaded as offline so that groupings can be changed around. It is a bag of bugs, I’m sure, but maybe it can be done on the side.

At any rate, I’m still busy with other things non-technical in nature. Truth be told, at this point I’m just more eager to use Janus for the stuff I’m doing than coding it.

Refining the definition of voice

I went one of the local libraries the other day with the intent of spending my morning there reading, and the rest of the afternoon drawing. As I moved closer to the book shelves, for one reason or another, I remained in the graphic novels area – a general genre of books that I have very little interest in. I still like some comic books: Asterix, Calvin & Hobbes, Far Side, Peanuts, TinTin, B.C., etc; I grew up with these. Thanks to my brother I was exposed to more serious minded comics like The Light and Darkness War, whose style and tone was, to a young teenager, something like a wide awakening – a baptism of imagination, if you’d like.  The panels were watercoloured, the lines were thick-thin, and the prose was poetic enough for me at the time to appreciate the authors’ collaborative voice.

I took some of this with me in my days in a fine arts college – my course was actually more oriented to graphic arts – and cultivated the inspiration as much as an under-achieving art student could. But diving into the labour force after college essentially marked the end of my adventures with serious graphic novels: comics which portray violence – often extreme – and gore, sex and explicit language. My foray into computer graphics turned my attention away from drawn stories.

It was only recently that, after longing to re-imagine myself and reassess my goals, I wanted to go back to basics and start drawing in earnest again, after ten years of professional CG work. And that’s where I found myself in front of the graphic novels area staring at a copy of DMZ. It featured mercenaries, and it described a dystopian landscape – I sneered on finding out that the story was set in NYC. NYC is over-mentioned, you see: ’nuff said.

I read the whole book. And despite the clumsiness of lines, the inexpert  description of form, and even the cliche of its language, I appreciated it mainly because it touched on subjects that I had swirling in my brain: guns-for-hire and wastelands of worlds. When I had finished I put it back on the shelf.

I wasn’t satisified: I wanted another to feed my mind. I laid eyes on one book. I wasn’t fully interested in the cover, so I scanned the shelf a bit more. I spent five minutes flipping through pages, then my eyes passed on the book again. I thought, “Hey, look, there’s a tired-looking Afghan with an AK47”. So, on account of sighting an AK47 I picked up The Photographer and started the story that would prove to be the closest thing to a second awakening.

The book is not a miraculous work. It’s not the most outstanding piece of literature I’ve read: I’ve been impacted more by other books, far deeper, and far longer. But this, being a graphic novel, it impacted me far more than any graphic novel I’ve flipped across.

As I read, it felt it difficult not to compare it with the recently-read DMZ. I became acutely aware of how dissatisfied I was with the other book, and how much more refined and expertly Emmanuel Guibert, the illustrator and visual director of The Photographer, had drawn, and – more surprisingly – had written the story of Didier Lefèvre. He had interspersed his comics with Didier’s photographs as he told of Didier’s travels to Afghanistan. Because the illustrations realistically depict people in a simplified (read: masterfully economical) style, the black-and-white photographs jolt you to the palpable sense that the story is really non-fiction. It doesn’t just fill in the blanks between the discrepancy that a photo-essay’s pictorial gaps suggests; it puts in a much more lucid level of storytelling.

Despite the fact that the subject matter was about doctors and a photographer, and that the only shot fired in anger was from an “asshole ‘muj'”, The Photographer thoroughly impressed, and more importantly, inspired me. In the CG field there is so much emphasis on detail, which involves real-world fidelity be it in photographic, or otherwise just physical terms. If it isn’t that, then it’s about resolving our work methods to conform to some pipeline: yet another detail to abide by. But within The Photographer, I found none of that. In the first place, the illustrations were minimalist, and yet I found excellent form posture, action, and human expression; detail was only necessary when it was necessary. In the second, the photographs were not used because they were photographic in quality, but because they, as photographs, told their part of the story. In the third place, the style of telling was uniquely individual: it wasn’t like one of those monthly high-profile, run-of-the-mill animated-feature-films (where almost every animated character motions with his hands and eyebrows the same (freakin’) way). The writing was powerful, brutally honest: honest enough that in two parts in the story it made me sad enough to tear up. A comic book has never had that effect, and I never thought it could have.

The Photographer’s impact on me as a reader and as an artist has caused me to re-evaluate what I am really about as a creative person. I would like to think I was a real artist, but I know that I am just a mercenary CG man with good artisan and technical skills. I’m not sure how much of being a professional contributes to creativity, but I can say that this book has forced me to confront the numerous habits and rules I have adopted as a professional. It is easy enough to say to think outside the square, than to realise that you are the square.

Rocketship: Launch Platform

The launch platform is slowly getting more elements in. I’m finding it a bit tedious, though it’s helped by some of the railing tools I’ve done.

The lighting is obviously a temporary thing – more of a hint than anything. The ‘tri-beam support’ is situated under the rocketship and is partially hidden by the platform structure. The picture below shows it by itself as a unit.

 

The platform, in general, has received smaller structures as well as some semi-completed buildings, which will be refined on a shot-to-shot basis if need be. Some camera movements have already been laid out, but there’s a lot of considerations still.

Tools

In the course of building the launch pad for my rocketship, I have built two very simple tools in Houdini, which are called HDA (Houdini Digital Asset). One creates simple straight ones that have options how high, how long, and the thicknesses of the top bar and middle bar, and how many vertical bars in the breadth of a metre. The other creates railings that follow the shape of a curve such as those found in staircases. These tools have been an immense help in laying down geometry quickly, and to control as many aspects of the object as I want.

While I find Houdini’s basic functions to be relatively awkward compared to more sophisticated modelling apps like LightWave or Maya, its true procedural workflow is such a mind-blowing tool in itself. Granted that it does take more time to do simple stuff, the benefits of making mass revisions on highly detailed meshes is easy-peasy. For example, the image pictured above includes one of my launch towers. Based on how I’ve set it up procedurally, I can not only specify how many floors I want on-the-fly, but I can just as easily change the height of each floor, and that, in turn, changes the geometry of the stairs to accommodate for the adjustment. If I had to do such adjustments in Maya or LightWave, it would be a rather tedious job, especially if I’ve made critical modelling choices beforehand, which usually happens anyway.

I think, overall, the significant thing about Houdini is that I’m having loads of fun again which I never had when I was learning Maya. I had fun learning LightWave and it was always a friendly package to use both as a hobby and professionally, but it has been increasingly a frustration as time goes on. I do wish that I could slow down a bit and start learning more about the tools and writing some Python in Houdini, but I do have a commitment; I have been turning my head away from tool-making tutorials and avoiding complex solutions. But hopefully, when this short is done, I can start delving into some of the workflow issues that have come up. I already have a list of tools I want to create.