Yellow is the New Blog

In response to: XPages - The Good, The Bad and the UGLY - II

I'm gonna have to disagree with both of you. JavaScript doesn't need classes because of closures... if you don't know what closures are, now would be an excellent time to find out. Nathan's spot on about Squawk: there's maybe a dozen lines of SSJS; and in every case it's just instantiating a Java object and calling a couple of its methods. But to be honest, all the apps I'm writing now include less and less custom Java classes because I'm sprinkling in calls to standard Java classes where I need them (same goes for @Functions, though that too is decreasing), and everything else is being handled in typical JS syntax. It's a very misunderstood language... far more powerful than its reputation (which is primarily the fault of lazy developers). Expect a followup post on my blog about what a JS "class" looks like. :)

I've occasionally mentioned the JavaScript concept of "closures", but recently it's come up in several conversations, so I wanted to revisit it in case any of you have been curious about what this is or why you should care.

Put simply, they're a complete departure from typical variable scope handling (unless you're used to Lisp or Perl). Consider, for example, the following LotusScript function:

Public Function getWidget (Byval widgetName As String) As Widget
 Dim localSomething As String
 Dim myWidget As New Widget(widgetName) 'Assume this class is defined elsewhere
 Let localSomething = "this is a pointless variable"
 Set getWidget = myWidget
End Function

Pretty straightforward: a new instance of some Widget class is constructed using the passed argument and returned from the function. A local String variable is declared and assigned a value, but it's completely unnecessary, because as soon as the function returns the new Widget object, any variables declared inside the function are garbage collected anyway. So if, for example, Widget contains a method that refers to the localSomething variable, that method will look for the variable within the class definition and, failing that, global variables... if none exists, expect an error.

Java adds another layer of garbage collection called block scope:

public Widget getWidget (String widgetName) {
 if (somethingIsTrue) { // assume a boolean defined elsewhere
  Widget myWidget = new Widget(widgetName);
 }
 return myWidget; // oops... NullPointerException
}

In this example, myWidget is actually local to the conditional (if) block: we can't refer to it outside of that block, because it's defined inside it. To refer to it after the block, we'd actually have to define it before the block, whether or not we plan to do anthing to it inside the block.

So what makes closures different? In JavaScript, an object returned from a function still has access to variables local to the function that returned it. "Whaaa?" Yes, the garbage collector knows not to destroy those variables until the object is destroyed, because it might want to refer to them later. For example:

var Widget = function(widgetName) {
 var localSomething = 'This may come in handy later.';
 return {
  displayLocal: function() {alert(localSomething);}
 };
};
new Widget().displayLocal(); // alerts the value of localSomething

In this case, an object is created that contains a single "public" method: displayLocal. This method can safely refer to the localSomething variable, because it's bound to that variable via closure. In essence, this allows JavaScript objects to contain "private" variables and methods: if we had defined any functions within the above function - but outside of the returned object - they could be called by the returned object... but not by other code outside of the function that contains them.

In other words, this allows JavaScript objects to be defined such that they behave more like class instances in other languages: they can contain public and private members. As such, in the above example, I "instantiate" a Widget via new... but worth mentioning is that I didn't need to: each time the above function is called, two new objects are created: a local String and an object containing a public method that can refer to that String even after the function that created both has returned. Hence, whether I include the new keyword or not, a new object with the same behavior template is created, and any locally defined objects bound to it by closure are new as well: a separate instance of localSomething is bound to each object created by calling this function. I tend to use new when calling functions that are structured this way to indicate (to myself and others maintaining my code) that the returned object behaves like a class instance, but the behavior is not dependent upon that convention.

One final syntactical nugget before discussing implications: tacking parentheses onto the example function's definition makes it behave something like a singleton:

var Widget = function(){
....
}();
Widget.displayLocal(); // we no longer need to call the function each time

In this case, the function is called immediately, so the variable Widget is assigned a pointer to the object that is returned from the function, not to the function itself. Widget IS the object. It has public methods bound to private data. For quite a while, I've preferred this approach to defining global singletons, because you don't have to call a function each time... you already have the object. But I'm finding that this tends to confuse people: they'll try to instantiate the object again, which causes the code to break, because Widget isn't a function, so treating it like a function returns an invalid result. In the end, of course, it's always best to keep global variables to a minimum; Yahoo recommends creating a single singleton that represents the entire application, and nesting objects appropriately within that space, and that's what I tend to do. This seems to mitigate the confusion, as most developers then grok that there's no need to construct new copies of the object; just "use" the object as constructed in-place, and customize it as needed.

So with all of that out of the way, why are closures actually useful? The same reason private members of a LotusScript class are useful:

• Internal storage of variables that cannot be mucked with from outside the object
• Minimizing global variables by associating groups of related data with a container object
• Probably most importantly, manageable code through proper decomposition without convolution of the public API

That last point probably bears repeating: as an object's complexity increases over the development (and enhancement) of the application or framework, you certainly could choose to make every method public. "Information hiding" (at least, in my experience) is more about shielding other programmers from the complexity of the code than it is about protecting the code from other programmers. When you're writing integration APIs (like a web service), of course, the latter factor increases, but the former is still crucial: imagine a WSDL that defined 237 service methods (and I assure you, they do exist)... it'd be a nightmare to keep track of what the service can be used for. Now imagine a complex web app that includes a JavaScript object with that many public members. It's highly likely that most of those members are for the internal convenience of the object itself, both to avoid duplicating the same code within multiple functions and to decompose the operation of each into tasks that are small enough to be easily understood. Closures allow for precisely that: decompose each function into smaller private functions, tuck away properties like internal counters that don't necessarily need to be publicly visible, therefore keeping the public API robust enough to meet the application's needs but simple enough to interact with and maintain.

In anticipation of the release of Domino 8.5, I've been familiarizing myself with Dojo, which thus far has mostly consisted of identifying syntactical differences between it and other frameworks I'm already familiar with. My conclusion is that it's not all that different; one of the two most noticeable differences is the inclusion of additional parameters in the actual markup to indicate to Dojo how certain tags should be handled. In this example, for instance, the author adds to a div a parameter of "dojoType" with a value of "dijit.form.DropDownButton": this tells Dojo that, upon load, it should render something that looks and behaves like a drop-down button where that div is placed, using the innerHTML of the first span it contains as the label for the button. That's fine. I like that: you can determine from the markup how the page should "look" to the user. Granted, I'd rather see them take a semantic approach, but meh.

The other difference, however, is simultaneously cool and troubling. Every "official" example or demo I've seen includes just the base dojo.js file in the markup, then in a custom script loads in everything else the page will need via the dojo.require() syntax. In the previously linked example, for instance, there are function calls similar to the following:

dojo.require("dijit.Dialog");

Why? Because the base Dojo file only includes core functions - such as XHR, array extensions, and the like... as well as a capacity for dynamically loading other script files. Rather than load the entire framework and only use a small fraction of its capability on any given page, dojo.require() allows you to tell the browser precisely what you need and, therefore, only load that. This should enhance performance by minimizing the amount of data the browser needs to download in order to provide the functionality your page requires...

...in theory. In practice, not necessarily. Some have compared this dynamic loading to Java's import statement, and there are certain similarities. When compiling a Java application, import tells the compiler which classes to load, and excludes everything else. Dojo also uses namespacing to provide clarity regarding what is being included. But the difference is that Java is compiled, and JavaScript is interpreted: when you click Save in Eclipse, therefore, it "builds" everything: the resulting class file is aware of everything it needs to be in order for your application to run. A web page, on the other hand, must download each file individually. As I'm sure I've mentioned before (ah, yes, here it is), a powerful way to improve the performance of any web page is to minimize the number of HTTP requests; although Yahoo's list of recommendations keeps growing, that's still their number one guideline. If, for example, you have 10 script files that are 20 KB each, and one 200 KB file that combines all of them, the browser will load the 200 KB file faster than it would load the 10 separate files, even though the same amount of data is exchanged. That's the primary reason I wrote JSFactory and still use it nearly every day (especially lately). Dojo, on the other hand, takes the opposite approach: I added the Dialog example to an application I'm working on (and no other Dojo functionality), and the result was 35 separate HTTP requests, ranging from 3 KB to a mere 84 bytes. I kid you not: one file it loaded was common.js:

({"buttonOk":"OK","buttonCancel":"Cancel","buttonSave":"Save","itemClose":"Close"})

That's the entire script file. That's ridiculous. I'm logged into Domino, so the LtpaToken cookie sent back to the server via a request header allowing it to decide whether or not I have access to that file is larger than the content sent back once it determined that I did have access to it. Dojo has a reputation for being slow, but it's mostly undeserved: I've found that, once the page has loaded, dijit responsiveness is roughly equivalent to widgets in other frameworks. But this dynamic loading that (I assume) is intended to streamline initial page load, in fact, has the opposite effect, and by the time the user is allowed to interact with the page, the impression of "slow" has already been established.

So what's a developer to do? Well, it's easy, actually... again, in theory. Dojo includes a custom build packaging system. So all you have to do is have a copy of the Dojo library on your local computer, write a build profile that designates which modules you would otherwise be importing via dojo.require() statements (heck, you can even include your own custom script files), and then run a shell script (or a batch file, for the majority of you who would be running this from Windows... though the tutorial assumes Linux and mentions the Windows option as an afterthought), and voila: you have a single (combined, compressed, and otherwise optimized) script file that you can then reference in your web page... assuming you've placed it somewhere on the server (either by adding/updating a file resource in a Domino database or moving it directly into the server's folder structure). Sounds a lot like JSFactory, doesn't it (other than automating the step where it updates your Domino file resource)?

But there are a couple minor caveats. First, you must have a copy of the Dojo source (which you can download here), not the built version that's automatically included on the server with Domino 8.5; second, any time the mixture of modules you need (or the content of any custom script) changes, you have to run the build again, and update the server's copy of the file. In other words, this can become very tedious.

Here's what I'd recommend:

• Use Aptana (or your favorite JavaScript IDE; mine is Aptana for various reasons, one of which is that it supports auto-suggest for Dojo and several other frameworks) to write all custom JavaScript code that doesn't need to be generated dynamically. This keeps your source in a folder the Dojo build script can reference.
  
• Create a build profile that includes a reference to all of the Dojo modules that you're using - plus your custom code - which you'll update whenever that module list changes. For bonus points, write an Aptana script that monitors all files in your project for save events and updates the dependency list automatically based on which dojo.require() statements it finds.
  
• If you're running Windows, place a .bat file for each project on your desktop that cd's to the Dojo build scripts folder and runs the build batch using your profile for the project. If you're using the Desktop toolbar, you can then update the build from anywhere with two clicks: one to view your desktop as a menu, and one to launch the build process.
• Create a project profile in JSFactory that includes just the build output file as the source. There's no need to specify a minified output, since Dojo will already have minified the source, but if you've configured your server to support GZIP, you can enable that option. Then with a single click in your Notes 8 sidebar, you can update the desired file resource element with the contents of your latest build.
  

At my earliest convenience (read: February) I'm going to update JSFactory to automate the middle steps, so your process would be: write the code in an IDE, click "Build Selected" in JSFactory... done. And just a reminder: if you've already bought JSFactory, you own it; assuming you kept the download link, it'll automatically point to the new version once that's released.

I was booking some travel recently and couldn't select a date on one of the travel sites because they were using window.open, so my browser blocked it as a popup. They weren't trying to sell me anything I wasn't planning on buying anyway, they weren't trying to install malware, it was just a date picker. But I couldn't use it without modifying my browser's security settings (or switching to a less secure browser). It's 2008... window.open? Seriously?

So, for any of you who still use window.open to display a date picker, or a comment form (see Andrew's blog as an example of the right way to do this), or... well, anything else... and the only reason is because you asked someone for an alternative and were simply told, "duh, use a positioned div" - but not told how - hopefully this post will show you how easy a better alternative can be.

To display something that looks like a popup but really isn't, all you really need is:

• a div that hides until needed and has a higher z-index than the rest of the page
• code to show and hide that div, and position it where you want it to display
  

What's a z-index? If you use layers in your Notes client development, you're already familiar with this concept: think of your user interface as existing in three-dimensional space. The x axis is horizontal, the y axis is vertical, and the z axis measures the conceptual distance between the "front" and the "back" of what's available to the user. Unlike x and y, however, which are typically measured in pixels or some other unit associated with the available width of the user's screen, the z index is theoretically limitless, and is simply identified by an integer. So on a web page (or a Notes layer), anything with a higher z-index is "closer" to the user; anything with a lower z-index is "further" away, and therefore displays "behind" the higher elements. Mozilla.org has a great tutorial on this concept if you want to dig deeper, but for the moment, just remember that the element with the highest z-index displays on top of everything else.

Here is an example (also available for download) of using a "positioned div" to display additional content that you might otherwise launch in a separate window with window.open. To enable this, start out by creating a div somewhere in your page markup, give it an id and a class (in my example, I'm using "pseudoWindow" as the id and "layerwindow" as the class), and define some CSS for that class:

div.layerwindow {
  background-color: #eeeeee;
  border: 3px solid #abcdef;
  display: none;
  overflow: scroll;
  padding: 3px;
  position: absolute;
  z-index: 2;
}

This causes that div to hide initially, as well as defining how it will look when it's no longer hidden.

Next, you'll need some JavaScript code for showing the div, positioning it, and hiding it again:

var LayerWindow = function(){
  var getViewPort = function(){
    var viewPortWidth;
    var viewPortHeight;
    var vYscroll;
    if (window.innerWidth) {
      viewPortWidth = window.innerWidth;
      viewPortHeight = window.innerHeight;
      vYscroll = window.pageYOffset;
    } else if (document.documentElement && document.documentElement.clientWidth) {
      viewPortWidth = document.documentElement.clientWidth;
      viewPortHeight = document.documentElement.clientHeight;
      vYscroll = document.documentElement.scrollTop;
    } else {
      var bodyTag = document.getElementsByTagName('body')[0];
      viewPortWidth = bodyTag.clientWidth;
      viewPortHeight = bodyTag.clientHeight;
      vYscroll = document.body.scrollTop;
    }
    return {
      w: viewPortWidth,
      h: viewPortHeight,
      yScroll: vYscroll
    };
  };

  return {
    open: function(id, options){
      var vp = getViewPort();
      if (document.getElementById(id)) {
        overlayer = document.getElementById(id);
        overlayer.style.width = options.w + 'px';
        overlayer.style.height = options.h + 'px';
        overlayer.style.top = (vp.yScroll + parseInt(vp.h / 2, 10) - parseInt(options.h / 2, 10)) + 'px';
        overlayer.style.left = (parseInt(vp.w / 2, 10) - parseInt(options.w / 2, 10)) + 'px';
        overlayer.style.display = 'block';
      }
      return false;
    },
    close: function(id){
      document.getElementById(id).style.display = 'none';
    }
  };
}();

Finally, update whatever would be triggering the window.open to instead just display the hidden div:

<a href="#" onclick="return LayerWindow.open('pseudoWindow', {'w':300, 'h':250});">Open layer window</a>

There are a multitude of ways to add further elegance and functionality to this (and nearly every widely used JavaScript framework has their own implementation) - for example, loading the content of the hidden div via AJAX prior to displaying it instead of using pre-populated content - but hopefully this will give you a head start toward eliminating your own use (if any) of window.open... please, think of the kittens.

Back when I was experimenting with XIDED,  I wrote a utility database to mitigate my primary annoyances when dealing with JavaScript - in Domino in particular, but also a few general pet peeves (you'll probably notice that the recently released beta of 8.5 addresses the first of these):

• The JavaScript editor in Domino Designer is an IDE pretending to be Windows Notepad, failing in that imitation only when it occasionally maintains the current indentation from one line to the next.
• As the complexity of a web application increases, so does the hassle of maintaining all of its application-specific JavaScript in a single source file... the ink was starting to rub off on my Ctrl and F keys.
• Conversely, if you split out your source into a bunch of smaller, more maintainable files, it's a pain to merge them all back into one, so the temptation is to just throw a script tag in for each one, which tends to negatively impact page load times (even if the browser is pulling each from the cache, it still has to check each one separately to see if it's in the cache, so it might not be downloading each again, but there's still a performance hit). But then you still have to refresh the file resource for each that has been updated.
  
• At one point, the combined uncompressed size of all of my source (including the frameworks I was using in addition to my application-specific code) was just over 1.5 MB, which is ridiculous. But minification and compression added two additional steps to my "build" process.

So out of pure old-fashioned laziness, I wrote what I now call "JSFactory": once I've defined a project version document (to allow multiple versions to be maintained in different directories), listing the location of the source files in the order that they should be combined, an output location for the combined file (and - optionally - compressed, and/or gzipped... since I've configured my server to deliver gzipped content), and a file resource element to update, I can refresh that element with a single click. So I can maintain lots of little source files, doing my coding in Aptana, and then update my application without even opening Designer. It works in Windows and Linux (and presumably Mac, though I don't have one to verify that).

So if I've had this for years, why am I just mentioning it now? Well, two reasons. The first is that Chris Toohey's SOTU Sidebar Widget Demo inspired me to add a similar widget to JSFactory, which I did yesterday:



The second is that I decided to try an experiment: in a singular (and possibly only) departure from my traditional "give everything away" approach, I'm offering this for sale. As its usefulness is restricted to a very specific niche task, and future versions of Domino may make it entirely obsolete (here's hoping, anyway), I'm only charging $9.99, and I didn't bother putting in a bunch of licensing restrictions; if you buy it, it's yours to use as you see fit: donate it to your whole team (assuming you have one), put it on as many workstations or servers as you wish, modify the source (yes, I left it open), etc. I realize this means that one person could buy it and just send it around to everyone else, but I'm not looking to "make my millions" with this... I've got a dream job. I just request that if you receive a copy without having purchased it and find it useful, consider buying your own license... it's the same price as most albums on iTunes.

Julian's post about  JavaScript stack traces reminded me of something I wrote late last year and had been meaning to mention but never did.

Web applications have steadily become more interactive over the last couple of years, and have been evolving from a collection of pages to a sequence of interactions and events. On the one hand, this can be very positive for the user, as we're no longer forced to trigger an entire page refresh for every mouse click. On the other hand, one challenge this presents is that many users are still in the habit of conceptualizing site navigation and interaction in a context of page history; in other words, they expect to be able to click something (i.e. a button or a link), examine the result, then click the Back button and see what they saw before... and click Forward to return. This was already a problem before Ajax when POST requests were involved, but adding Ajax to the equation can compound this even more: by default, not only will clicking Back not show the user a previous state of the current page, in a "one-page application", it'll actually exit the application, possibly even your entire site.

So how can we get around this? Ironically, most workarounds I've seen (including the one that I implemented and am about to describe) make use of the HTML element that was often used to provide Ajaxy behavior before what we now consider to be "true" Ajax gained widespread use: namely, the iframe. When an iframe's location changes, an entry is added to the browser's history (in some browsers... more on that later), even though the location of the page containing it has not changed. As a result, clicking the Back button will simply return the iframe to its previous location without modifying the container's location. Similarly, once that's occurred, the history contains an entry in the forward direction, so clicking Forward at that point will again navigate only the iframe, still leaving the container location intact.

How does that help us? Chances are, those familiar with JavaScript's Function.apply() have already guessed where I'm headed with this. As I commented on Julian's post, a handle on a function can be passed as a parameter to another function. This is used all the time in Ajax requests to specify a callback function: once the request has been processed, the specified callback function is executed to respond to the result of the request. In this case, however, we're not associating a function with a request for remote data (or submission of data); instead, we're actually associating a function (and the parameters that should be passed to it) with navigation. Here's how it works:

• Instead of calling the function directly, we pass a handle on the function (along with an array of parameters to call it with and an optional scope) to another function - AjaxHistoryManager.register(). For example:
  

  function sayHello (sender, recipient) { this.innerHTML = recipient + ', ' + sender + ' says hello.'; }
  AjaxHistoryManager.register(sayHello, ['Me', 'World'], document.getElementById('helloDiv'));

  
  NOTE: the lack of parentheses after the reference to sayHello is intentional; we're not calling the function at this point, just passing it as an argument.
  
  
• The register() function creates an object to store the function handle, parameter array and scope object (which defaults to the passed function if no scope is specified), pushes that object to an array that stores all of the function calls that have been registered since the container page was loaded, and increments a counter.
  
• Finally, register() sets the source of a hidden iframe to the URL of a Page design element (assuming you're using Domino; a very minor tweak to this would allow it to behave identically in PHP, ASP, JSP... pretty much anything), including a query string parameter indicating the current counter value.
• The Page loaded in the iframe has only one job: it includes a script tag that (using computed text) passes the counter value to window.parent.AjaxHistoryManager.execute(); this function retrieves the object stored earlier and calls the associated function with the stored parameters, binding "this" to the correct scope object. The reason I almost never use "this" in JavaScript is that any number of factors can cause a function to lose a handle on the scope you originally intended it to have, which will cause any references to "this" to apply to the new scope. That's not a problem in this case, because we're calling .apply() on the function to be run, which allows us to explicitly specify a scope object. So, in the example above, "this" refers to the HTML element that was passed as the scope object. If the function you're registering doesn't have any references to "this", you can leave off the scope parameter entirely and just pass the function handle and a parameter array.
  

The result is that the registered function is still called almost immediately (depending on how long it takes your server to send the 94 byte overhead for the iframe Page), but the browser history now contains an entry associated solely with that event. Ergo, executing a series of events in this manner allows the user to execute them again (with the same parameters and scope each time) simply by clicking Back and Forward.

You can see an example of this behavior here (the sample database is also available for download). On that page, click the "Increment Counter" button a few times, then click Back and Forward (Alt+Left / Alt+Right also work)...  the displayed counter will increment and decrement based entirely on those navigation events. This works correctly in Firefox and I.E., but not Safari or Opera. In Opera, navigation does appear to update the iframe, but the stored function is not called; in Safari, clicking Back actually navigates the entire container page. In both cases, though, the function is still executed initially upon registration; in other words, your application will still work, but will not have the benefit of triggering previously executed functions again via navigation events. I still have never had to support either of these browsers within an enterprise context, but if anyone has ideas on how to extend this approach to fully support either or both, let me know.

One last disclaimer: this approach is best suited for "read-only" operations. For example, user interaction causes some change to a portion of a page (i.e. resorting data), and we want reverse navigation to "undo" that change. This approach gets a whole lot messier if a POST is involved: if the methods being executed are actually updating data in the database, then executing them in reverse order needs to revert the database content to its previous state - whether by resetting field values on one or more documents, or actually removing documents that were added by the executed method(s).