Baconbutty - Programming

I have been playing around with Nest a little further, and have produced a version 0.2 which provides for asynchronous loading using the script element, and a slightly improved system for specifying namespaces and requires.

It is still very rough around the edges, and is really just a work in progress. I am aiming for a 0.3, including XmlHTTPRequests and possible optional compliance with some of the CommonJS standard.

The use of dynamic script loading may give IE something to choke on (horrible message bars for your users) if security settings are high.

Does this all have any use at all? Probably not. See my comments at the end. I have read quite a few web pages on the subject of ECMAScript modules, and the upshot is, there are a lot of views, and a lot of great, efficient, and implementations, so there is no need for yet another module system.

Neverthless, in the spirit of play and exploration, I have created my own more complex module loader, which should be fairly extensible, but is far from being elegant ECMAScript code.

The Code

If you want to get straight to the code, then you can find it in these files:-

nest.0.2.js - the plain source code
Nest 0.2 Test - a test file
Nest 0.2 Browse - a browsable file of the source code.

Ironically, when compiled with Google Closure Compiler, it compiles down to 0 bytes, because it exports nothing!

Nest Instantiation

Nest requires the core "nest()" function. You can instantiate a nest instance any way you like. Just fiddle around with the code! The basic way is new nest() or nest(). The latter will instantiate and return an object without the need for new.

var n = new nest();
var n = nest();

You can create multiple instantiations, and load modules separately into each.

Module Construction

Before we look at module loading, first what a module looks like.

Minimum Requirements

Well, there are no requirements, except that a module

must be in a named function declaration or named function statement
must specify a namespace
may specify other modules it requires (or you must load these separately yourself)

Source Code Example

So what does a module's source code look like?

You have several options, shown below:-

Option 1 - a simple global function declaration or string containing a simple function declaration.

function app()
{
	namespace : "nest.apps.test";
	requires : "nest.lib.js";

	function ExportFunction()
	{
		alert("Hello from app");
	}
	exports(ExportFunction);
}

var app = 'function app() {  namespace : "nest.apps.test"; requires : "nest.lib.js"; function ExportFunction() { alert("Hello from app"); } exports(ExportFunction); }';

The purpose of storing the module as a string, is to save on mutliple parsing. I.e. first parsing the function module, and then reparsing when it is inserted in an eval in nest. Thanks to David Flanagan for pointing this out.

You will note that I use a label followed by a string, for configuration purposes. I.e. namespace : "nest.lib";.

I was going to use a simple string "namespace nest.lib"; but Firefox (the only browser to do this) strips out these types of statements unless they are attached to a label in Function.prototype.toString();.

Option 2 - a descriptor object, to provide the namespace and requires information externally

var app = 
{ 
	isNestModule : true,
	author : "",
	version : "",
	identifier : "app",
	namespace : "nest.apps.test",
	requires : ["nest.lib.js"],


	// Function or String - your choice - even something with Deans packer 

	source : function app() { function ExportFunction() { alert("Hello from app"); } exports(ExportFunction); }

	source : 'function app() { function ExportFunction() { alert("Hello from app"); } exports(ExportFunction); } '  

	source : eval(function(p,a,c,k,e,r){e=String;if(!''.replace(/^/,String)){while(c--)r[c]=k[c]||c;k=[function(e){return r[e]}];e=function(){return'\\w+'};c=1};while(c--)if(k[c])p=p.replace(new RegExp('\\b'+e(c)+'\\b','g'),k[c]);return p}('0 1(){0 2(){3("4 5 1")}6(2)}',7,7,'function|app|ExportFunction|alert|Hello|from|exports'.split('|'),0,{}))

};

Option 3 - Either of the above two as a property of the nest function, if you don't want to clog the global namespace.

nest.app = function app() { "namespace nest.apps.test"; "requires nest.lib.js"; function ExportFunction() { alert("Hello from app"); } exports(ExportFunction); } ;

nest.app = { 
	isNestModule : true,
	author : "",
	version : "",
	identifier : "app",
	namespace : "nest.apps.test",
	requires : ["nest.lib.js"],
	source : function app() { function ExportFunction() { alert("Hello from app"); } exports(ExportFunction); }
};

Be carful if you have a named function statement, and want to refer to the function it it by name. The following is the case in IE.

var f = function exports() { alert(f === exports); } 
f() // alerts false

Module Naming Convention

A module file name must be in the form ([A-Za-z0-9]\.)*[A-Za-z0-9] and must end in .js.

E.g. dom.js or nest.dom.js.

In otherwords it can contain any of A-Z a-z 0-9 . but not _.

A module function name must be the same as the file name, without the .js and with other . converted to _.

Thus for a file name nest.dom.js the corresponding function name would be nest_dom.

As you can see, it should at least form a valid ECMAScript identifier after this transformation (e.g. it cannot begin with a digit).

Module file names are otherwise at your complete discretion, and they do not dictate the namespace in any way.

Namespaces

A module may specify a namespace, which must again be in the form ([A-Za-z0-9_]\.)*[A-Za-z0-9_]

In other words it can contain any of A-Z a-z 0-9 . and this time also _.

All namespaces begin from the root new nest() object, and internally within the modules start with nest.

If you specify a namespace without starting with nest, then nest is implicit.

You are free to specify any namespace for the module, and nest will create all intermediary objects as required.

There is no requirement for the module file name or module function name to correspond to the namespace. The namespace can be anything.

If you do not specify a namespace, then your module properties will be added to the root nest namespace.

The namespace for your module can either be defined inside the module function:-

function module()
{

namespace : "nest.module";
//EQUIVALENT TO
//namespace : "module";

}

or in a descriptor objcet

var module = {
	isNestModule : true,
	namespace : "nest.module"
};

Requires

These are purely optional. You can always manage them manually.

You must ensure that there is no circularity, two modules cannot be dependent on each other.

A require is either:-

A file ending in .js
A module name (pointing to an already loaded module)

When you specify a require, nest will first look for an already loaded module with a name corresponding to the require name.

A file can be a relative path, or an absolute path.

An absolute path can be:-

An http scheme URL http:// or https://
A file schme URL file:///
Local file path

E.g. require : "http:www.baconbutty.com/dom.js";.

E.g. require : "c:\dom.js";.

E.g. require : "js/dom.js";.

E.g. require : "dom";. // already loaded module

Exports

Within each module there is available the nest exports function. You could just add the functions to the module instance object, but exports() lets you do some pre-processing first if you want to.

Module Loading

Options

The Nest system lets you load your modules on one of two ways:-

Everything synchronous - this assumes that all your module source code has been downloaded.
Everything asynchronous - if the source code of just one of your modules or its dependences still needs to be downloaded, then everying must be done asynchronously - with liberal use of timeouts.

Interestingly, in passing, I found a way to completely crash windows using IE7. If you create a huge number of setTimeouts, my system just blacked out, no warning.

Simple Synchronous Manual Loading

The simplest way to load modules, is to directly load available modules in dependency order (if you know this already):-

var n = new nest();
n.loadModule(core);
n.loadModule(util);
n.loadModule(dom);
n.loadModule(myApplication);
n.myApplication.main(); // whatever is your start function

This can only work synchronously, and all module source code must be pre-loaded first as a series of prior script tags, before the above is run.

Any requires will be ignored, because you are already loading in the correct order.

Run an Application Synchronously

If you have all the module sources downloaded, and they all specify their dependency chains, then nest will do the above automatically:-

nest = new nest();
nest.run(app, "main");

When all the dependency chain for app is loaded, this will call the exported main function of app.

Run an Application Asynchronously

If any of your module sources still need be downloaded, then runAsync will attempt to download all the dependencies, and will queue them until all are available:-

nest = new nest();
nest.runAsync(app, "main");

Currently it will wait for 30 seconds for a module, before throwing an error and aborting the load.

Module Systems

Fundamentally I guess a complex module loading system such as this isn't really necessary, as for maximum performance, you are going to be packaging up all the dependencies in a nice minimised and compressed file. In fact, I am not quite sure what it is good for. I have noted some web pages that require statements are really just clues to some form of JavaScript compressor / minimiser, like C linking.

It seems that the problem with ECMAScript module systems, where people struggle to agree, is that the adaptability of ECMAScript means that it is trying to be all things to all men, and there is perhaps no one module system to rule them all (and in the eval, bind them!):-

1. There is the traditional approach (Operating Systems / Servers / Java / .NET) which is to have a large quantity of pre-installed components and libraries, which your final application synchronously calls on on the same computer. This is where ECMAScript servers are at I guess. This is easy to achieve for servers (you control it), but for client computers, this may work for Operating Systems, Java, and .NET, but it is far from clear it will work for ECMAScript, and it does throw up issues for developers about ensuring that the client environment has the right version on it (.NET suffers from this particularly). This is unlikely to ever happen in a generalised way for ECMAScript libraries any time soon, given the large numbers of competing libraries, and any attempt to impose anything other than a loose iteroperability requirement between libraries would risk creating a proprietary lock-in.

2. As a derivation from the above, there are add-ons to the browser environment (Silverlight, Flash, JavaFX, Air, Google Gears etc) which are smaller initial installs and remarcably deliver massess of functionality. Again, as with 1, it is unlikely that we are going to agree on a single ECMAScript browser add-on which contains an agreed set of common ECMAScript library code that extends the build in features of the language itself.

3. There is then the ECMAScript self-sufficient web application, which is a web focused application (e.g. a mail client) which may use limited amounts of 1 and 2, or none at all, and works through a combination of its own ECMAScript libraries and standards based browsers and their standards based object models. For these, as they are visited repetitively, there is a need for efficiency, and as it is difficult, I guess, to arrange for clients to first install one-off downloads of you full ECMAScript library which can be re-used for later visits, for most web applications, a client-side developer will want to have all the scripts and their dependencies pre-packaged, minimised, and ready to go at development-time, rather than having it all being resolved at download-time. I.e. as a client-side developer, for your module system, you perhaps want something more like a C compiler and linker, which creates a nice efficient package from all your libraries, which can be downloaded synchronously up front. I think this is something the likes of dojo, YUI and Closure already offer in spades.

4. Then there is the stand alone ECMAScript application (not sure how much of this there is yet), which is designed really to be a desktop application. E.g. I have developed a time recording system, which just happens to use HTML and JavaScript, rather than .NET or Java or Cocoa. This runs in Prism or as a .hta. Here, as it is a single download (rather than repetitive revisiting in 3), the client is likely to cope with downloading all the libraries that come with it, and it throws up fewer server efficiency issues as you are doing no more than you would if you were delivering a normal client .exe file. Here you could either throw in all your libraries (whether used or not) or go down the C linker style route, and package your app with only what is needed.