Olov Lassus

Sane TRIM on Mac OS X

2015-06-04T13:35:00+02:00

Using a third party SSD on your Mac while trying to keep TRIM enabled kind of drives you crazy, and OS X Yosemite didn’t help. Here’s my procedure for keeping my SSD reasonably trimmed while keeping my mind reasonably sane. It may work for you too.

Find an external disk, preferably a 2.5” one. If you’re like me then you have a bunch of spare 2.5” SATA disks on a shelf somewhere - pick a lucky winner and buy a $10 USB enclosure for it.
Install OS X 10.8 (10.9 may also work well) on the external disk. Boot it up (hold alt upon boot to select it). Disable automatic OS X updates. Keep it offline because it is (or will become) obsolete in terms of security updates.
Enable TRIM via the hack you know of already. It doesn’t really matter how you do it - the “TRIM Enabler” app will work fine. Reboot and verify that System Report says “TRIM Support: Yes”.

Once every 1 to 3 months, you’ll boot your Mac with the USB disk. Open Disk Utility, select your SSD partition(s) - do not select the entire disk - click Repair Disk and wait a minute or two. The log should say “Trimming unused blocks.” You can reboot back into your main system immediately. The whole procedure should just take a couple of minutes.

I found this to strike a nice balance between sanity and perf. Oh and yes, any TRIM-aware SSD will benefit from this treatment - however great its GC may be. Good luck!

Little dunder proto

2013-10-15T13:50:00+02:00

Hi, this is your son’s school. We’re having some computer trouble.

Oh dear - did he break something?

In a way. Did you really name your son __proto__?

Oh yes. Little dunder proto, we call him.

Well, our administrative web application is completely broken now. I hope you’re happy.

And I hope you’ve learned to use stringmap.js because objects aren’t stringmaps.

Also stringset.js. They resemble ES6 Map and Set API-wise, add minimal overhead and work everywhere.

In tribute to xkcd 327 - Exploits of a Mom.

Show comments (Hacker News)

ES3 <3 block scoped const and let => defs.js

2013-05-04T17:29:00+02:00

I released defs.js at the Front-Trends conference last week. Defs transpiles ES6 block scoped const and let declarations to beautiful ES3 code using vars. It’s also a pretty decent linter when it comes to variable and scope analysis.

ES6 block scoped let and const

Let me give you some background on why I created Defs. One of my favorite features in the upcoming ES6 language is block scoped let and const variable declarations (harmony wiki page). A while back I decided to start using them in my back-end code. This is possible thanks to node --harmony. Run it like so, make sure your code is in strict mode and off you go! With that a large JS annoyance (function scope) was gone with the wind, and I got to use const as a bonus - which turned out to work remarkably well with my JS style of programming. Nowadays mostly all of my node code is written in ES5 + cherry picked ES6 const and let features. Let’s call this constlet style JS for now.

constlet style JS is great, and I highly encourage you to try it out. It makes me much happier as a programmer and it makes my code much prettier and easier to reason about. Are you still reading? Up with a console and try it out in node! Then come back.

const is the norm

That block scoped variables is an awesome thing is no news for those of you that have a {} background. let is a better var in every respect so I won’t pitch it further. let is awesome. const, on the other hand, may seem unimpressive at a glance. If you’re not used to it then it may seem like something to use for the rare occasions when we define a numerical constant such as const godspeed = Infinity. But it’s so much more than that.

A striking thing that happens when you start using const is that you realize that the vast majority of your old vars were initialized but never modified (don’t confuse this with deep immutability which is a different story). Non-mutating variables are the norm, so they should have the default variable declaration keyword - which is const in ES6. For those variables that you modify, be it accumulators, loop indices or whatever - you make those stand out by using let. Once you get hooked on this style it’s impossible to go back. Ask any Scala-slinging friend of yours about val (~const) and var (~let). There are people swearing by this in Java-land too, finalista’s I’ve heard some of them call themselves because of the final keyword they have to add everywhere. To give you an idea of how common const is: The source code for defs.js has a const coverage of over 80%, which means that I used 4 consts for every let.

let is not the new var, const is.

Defs transpiles to beautiful ES3

So. Use constlet style JS and be happy, right? Except for the browser. We need to be able to run our client-side code in current browsers (so ES3 or ES5). Until now there was no decent way of transpiling constlet style JS back to ES3. Traceur and others use a style where a myriad of try-catch statements are generated which is as bad on your eyes as it is on your VM. This is no criticism on Traceur which is a very ambitious project with a lot of features to cover. Traceur will surely improve also when it comes to block scope.

So I created defs.js, a transpiler that takes constlet style JS and spits out plain-old ES3 code, as beautiful as its input and maximally non-intrusive. It works by doing static analysis, changing all const and let declarations to var, and an occasional variable rename here and there. The output looks more or less identical to the code you would have written with vars in the first place. Same line-numbers, easy debugging and no lock-in shall you decide to go back to the world of vars. Because Defs needs to do a bunch of scope analysis anyways, it ended up also being a pretty decent linter when it comes to variable and scope analysis.

Try Defs

Install Defs by npm install -g defs, then run defs mycode.js and there’s your transpiled code on stdout or your errors on stderr. There’s more info about configuration such as how to tell Defs about your globals (if you want it to analyse those) in the defs.js README.

Want to toy around directly? Bring up your browser’s JS console and copy and paste this: defs("const x = 1").src. It should give you "var x = 1" in return. Try defs("const x = 1; { let x = 2; }").src, which should return "var x = 1; { var x$0 = 2; }", a rename example. Try more advanced examples. Try modifying a const, try creating two consts (or lets) with the same name in the same block scope or try accessing a const or let before it has been initialized.

I know of only one use case that cannot be handled with renaming, namely loop closures - i.e. a closure that references a variable that is declared outside of its own scope but inside the loop. An example would be:

for (let x = 0; x < 10; x++) {
    let y = x; // we can skip this when v8 issue 2560 is closed
    arr.push(function() { return y; });
}

In this ES6 code let y is a new binding per iteration while a transformed var won’t be. If you run this through defs it will refuse to transpile it and give you this error:

line 3: can't transform closure. y is defined outside closure, inside loop

The solution is to do as we already to in our current ES3 or ES5 code, create the binding manually:

for (let x = 0; x < 10; x++) {
    (function(y) {
        arr.push(function() { return y; });
    })(x);
}

Defs will happily transpile that.

Toolchain and convenience

I can already hear people saying that they’d like to try this kind of thing but that the save-and-refresh development cycle is precious. Fear not. I guess that most aren’t running their code locally using file:// but use a local HTTP server, in which case you may have the opportunity of hooking a toolchain (including Defs) into it. If you serve using node this should be very simple.

It’s even simpler if you use Chrome. Go to chrome://flags, check Enable experimental JavaScript and you’ve got the same support for const and let as you have with node --harmony. Just remember that you must "use strict" for it to work.

Defs works great with James Halliday’s excellent Browserify. Here’s how I create the defs_bundle.js file that’s included in this page (view source):

browserify -r "./defs-main" > defs_bundle_constlet.js
# defs_bundle_constlet.js contains defs.js and all its dependencies

# defs-config.json:
# we allow duplicated vars because of defs dependencies
{
    "environments": ["browser"],
    "globals": {
        "define": false,
        "require": true
    },
    "disallowDuplicated": false
}

defs defs_bundle_constlet.js > defs_bundle.js

Things will be even easier once we have a Browserify plugin for Defs.

Underpinnings

Defs uses Ariya Hidayat’s Esprima parser to create an AST from source code. Defs performs static analysis on that AST and spits out errors or transformed source code. Static analysis is the hard part of Defs’ work, the actual source transformation is pretty trivial after that and it can be done in one fast textual shot using alter.js. If you wonder why I didn’t use JSShaper to create Defs then it’s for exactly that reason. Defs need a rock-solid and well-maintained parser. Esprima is that, my forked version of Narcissus I used for JSShaper is not. Thanks for Esprima Ariya, I have a couple of constlet parser bugs coming your way soon. :)

Scope of the Defs project

The scope [sic] of the Defs project is small on purpose. It transpiles const and let back to ES3 in a maximally non-intrusive manner, it performs static code analysis related to scope, and nothing else. Defs is not likely to take on other ES6 features.

Presentation

Feel free to check out my Front Trend presentation LET’s CONST together, right now (with ES3) for more info about Defs.

License

MIT because others are generous so I want to be generous too.

Try it, break it, help fix it don’t hate it

I hope that I triggered some interest in checking out what let and const could do for your code base. Use it with node --harmony and then consider using Defs to bring the same features to your client code. Check out the github project and npm package.

Let me know what works and what doesn’t. File issues or even better send pull requests. JS needs <3 and so does Defs. Peace and happy hacking! \o/

Show comments (reddit)

Taking the leap

2012-06-06T23:40:00+02:00

I’m taking the leap. I’m starting my own software company.

I’ve been working with IP-TV set top boxes for the last 6 years; first with Kreatel Communications, then aquired by Motorola, then split into Motorola Mobility and now recently acquired by Google. I came to Kreatel as a software engineer and since two years back I’ve had the luxury of leading our Technology Group, meaning our research and prototyping work. It’s been great, and it still is.

In my previous life before all this I did computer and console games, starting with the 3d shooter SubStation for Atari STE two decades ago and followed by PC, PSX and XBOX development full time a few years around the millenium.

I then attended the Linköping University (one decade ago, in 2002), and got my M.Sc. four years after. I learned tons there. It brought me to Lisp, gave me a deeper understanding of hardware all the way down and allowed me to experiment with CPU & computer design/implementation in FPGA’s. And I finally really learned the maths behind the 3d-games I used to write.

My finishing stretch of the University also came to define my spare time programming to come. I got into JavaScript by accident while researching how to create richer programming environments that turns the live-knob to eleven (where Smalltalk, Lisp Machines and modern edit-and-continue are well below ten), see Describing Live programming (again). I’d love to revisit that topic some day.

My more recent spare-time programming led me to creating Restrict mode for JavaScript and the JSShaper source transformation framework, a year and a half ago. The big difference here compared to my previous spare time projects and ideas (that mostly remained ideas) was that this time, I decided to just do something, release the code as permissive open source, and see what happens. Testing the do good and good will come to you thesis, if you will. Somehow that must have worked because I got invited to speak at DevLin, Front-Trends and Webshaped all in the last three months. I met a lot of interesting people there and I got to speak about things I’m deeply passionate about. In my book that’s as good as it gets.

I should mention Dart too. It resonates with me a lot, obviously when it comes to certain aspects of language semantics and easier to reason about programming, but more importantly in the direction I believe that they intend to push their development environment. There is a lot of Live Programming like thinking there. Gilad Bracha’s recent example of creating a test that fails with a noSuchMethod, then gradually filling out the implementation without ever stopping the program - while the context is right in front of the programmer, is similar to one I gave in my thesis. So yeah - it resonates with me, a lot.

With the same idea of just doing something to see what happens, I got involved with Dart early on. I thought of creating a JSShaper for Dart but got distracted partly by an increased community interest in my JS projects. So far my most useful contribution to Dart is a syntax highlighter. Speaking about Dart. I’ve had the opportunity to meet with a few members of the team, including Peter Ahé, Mads Ager, Kasper Lund and Lars Bak. It’s well known that they are VM experts but I quickly realized, especially so during discussions with Peter, that their ideas about what programming should be expands well beyond VM’s and language semantics. I hope to be able to play with a Live Programming like Dart environment one day. Three more things about Dart. (1) Any community should have a Bob Nystrom. He’s the money. My wife quotes him now and then (“I feel all giggly inside” - regarding the syntax highlighter running live at Github), which is kind of weird. (2) The haters got at least one thing right: Dart isn’t a silver bullet for everything web, because it doesn’t have a great JS interop story yet and perhaps never will. (3) If you think that Dart is mostly trying to solve a performance issue, think again.

By the way - I can talk/rant for hours about programming environments and what programming today is vs what it used to be decades ago already, and where we could go should we just prioritize it. Invite me and I’ll come talk about it. Or programming from an “easier to reason” about perspective. Or restrict mode and JSShaper. Or making superfast sprite routines for the Atari STE by generating op codes from pixel data. I like conferences!

And now, I’m starting my own software company. I’m lucky enough to have Motorola wanting me to stay part time, allowing me to keep doing the things I love there while at the same time taking a leap to grab a specific opportunity with my own company.

What I’ll do? A software product, not related to anything above. It’s for companies within a niche market, where it should simplify things and increase productivity a lot. So in that way similar to restrict mode and Live Programming, but not programming. It’ll be a web app and written in JavaScript using restrict mode. I don’t know where it’ll take me but I’m putting my bets on do good and good will come to you once again.

I’m taking the leap, and I can’t wait to get started.

JSShaper grows up but stays fit and fun

2012-05-22T00:42:00+02:00

I created JSShaper a year and a half ago because I needed a way to do source-preserving JS-to-JS transformations so that I could implement restricter, the Restrict mode for JavaScript checker. There was nothing else out there so I ended up creating Shaper. Some of the defining (and often unique) features of Shaper at the time was its simplicity of AST traversals (one function, not dozens of visitors), its precise weaved token and node representation that facilitates repeated tranformations on the node level at O(1) complexity, its declarative matching based on templates and wildcards that lets you describe the template as JavaScript code but still matches on the node level, its corresponding declarative replace function that lets you rewrite fragments of a tree without knowing the details of its inner property-structure, the smart and mostly closure compiler compatible binding of comments to nodes, the notion of pluggable Shape(r) pipelines, and more. Those are still defining features, and still often unique. While I created Shaper mostly for restricter, I decided to write a bunch of other plugins as well to show as examples if nothing else. One of the simplest of these ended up being one of the most useful, namely asserter. Check out C-style assertions in JavaScript via JSShaper for more information about it.

I wanted to share with you some recent good stuff happening in Shaper land.

C. Scott Ananian has created the most advanced Shaper plugin yet, yielder, which transforms yield generators (JavaScript 1.8) into plain ES5. Impressive work. I hope to be able to merge it into Shaper master soon, but for now use cscott’s repo.

cscott also contributed a number of fixes to Shaper, imported a newer version of Narcissus and rebased all my modifications on top of it. If you find any valid JS that Shaper fails parsing or round-tripping (node run-shaper.js yourfile.js --source) then please open an issue and we’ll fix it.

Finally, cscott performed AMD-fication surgery on the Shaper code base. I initially supported running Shaper through any shell you could think of, using the technique described in JavaScript modules in shells, browsers, everywhere. While cute, it was cumbersome and we decided to drop support for anything but node.js and the browser. AMD was a nice match; it works great in node and great in the browser, and the r.js tool is more convenient then concatenating files together. I also want to mention that Jez Ng participated in pushing node-compatibility forward, although that patch never landed. Another one of his patches did though.

Noah Sloan independently created jmvc-3.2-upgrade-assistant, a Shaper based script for helping upgrade from JavaScriptMVC 3.1 to JavaScriptMVC 3.2. Looks awesome!

I was at webshaped.fi last week presenting JavaScript: the subsets we use (including Restrict mode for JavaScript and a little bit of Shaper) and met a guy who who told me that his company uses Shaper similar to Noah, for automatically migrating code between different framework versions. Whoever you are I’m terrible at names so please ping me on Twitter and let’s keep in touch!

I gave the same talk at Front-Trends the month before, which was my international conference speaking debut. Both talks were recorded so I’ll post the links once available. If you want me to come talk somewhere about coding with a focus on “easier to reason about”, restrict mode, Shaper or something related just contact me! I can easily do an hour presentation about Shaper and the concepts behind it, for example.

Creating Shaper plugins is very fun, and not that hard. Try it! To get started I recommend checking out the sources for asserter and stringconverter (simple transformational plugins) or bitwiser (simple static analysis plugin). You’ll find these in the plugins/ directory. Then check out everything in the examples/ directory as well.

Thank you for trying out, contributing to, writing about or filing issues on JSShaper.

Peace!

Show comments (reddit)

Call for testing: Pygments Dart highlighter

2012-04-22T13:30:00+02:00

As mentioned before, the Dart highlighter lives in the upstream Pygments repository now and is included in releases since 1.5 (released Mar 10, 2012). I will try to keep it correct and updated.

Another milestone is that Github recently started using it! I found a few bugs just by clicking around in people’s repositories. The most obvious was keyword-detection on substrings, i.e. the String substring of StringBuffer got marked as a keyword. Another was failing to recognize escape characters inside string literals. Both are now fixed in my repo, so the example below renders fine.

main() {
  StringBuffer sb; // String isn't a keyword
  var integer; // nor is int
  var str = "Escapes \"just works\" now";
}

I need help with testing the highlighter to find bugs or missing features. I’m maintaining the Dart highlighter in my bitbucket repo. You can help by downloading it, trying it out on your own code and letting me know what works or not:

hg clone https://bitbucket.org/olov/pygments-main
cd pygments-main
./pygmentize -f html -o yourfile.html yourfile.dart

If you don’t have Mercurial (hg) installed then you can also click “get source” on bitbucket to download the source as a zip-file. Running pygmentize will create markup from your source code, which you then need to combine with a style sheet to get some color going. Feel free to copy my color scheme (syntax.css) if you like it. You can also run pygmentize yourfile.dart without -f and -o to get highlighted text directly on your console.

To install the development version of pygments system-wide just run sudo ./setup.py install.

File issues on bitbucket or fire a mail to the Dart misc thread or me directly. Even better, submit a pull request! :)

Show comments (Dart misc)

Meteor meets NoGPL

2012-04-13T09:30:00+02:00

Update 2012-04-21: Meteor has switched to the permissive MIT license.

Meteor made a big splash this week. Check out their screencast if you haven’t already. For “real-time” web applications it really seems to simplify things a whole lot. The team behind it seems plenty experienced and they are building a business on Meteor by going with the GPL/commercial dual-license approach made popular by MySQL and Trolltech/Qt. The basic idea is that the source code is publicly available for people to look at and modify and that it costs nothing to use as long as you’re building a GPL application with it and give out your source code too. Alternatively, it can be licensed commercially like any other proprietary product. Dual-licensing is not the only way to monetize open source development - a popular alternative is to offer support, training and consulting, like 10gen does for mongoDB. Anyways, I think Meteor is plenty cool and I wish them the best of luck!

The web development community hasn’t really taken GPL libraries/frameworks to heart, neither client- nor server side. Let me elaborate on that.

Check out the most watched JavaScript repositories on Github. Every single one on the first page is available under a permissive license. In a nutshell permissive licenses often require attribution but never require source code redistribution. Popular permissive licenses are the 2/3-clause BSD, MIT and Apache 2.0. Contrast this with the GPL whose copyleft nature spreads to derived works. Back to those popular JavaScript repos. The list includes Node.js, Bootstrap, jQuery, Backbone.js, Modernizr, Three.js, Express, Socket.IO and plenty more. Node.js is built upon V8, OpenSSL, libuv, libev, libeio and other libraries. Permissive. So is the vast majority of the NPM ecosystem.

Let’s check out some server-side frameworks in other languages. Rails, Django, Tornado, Play, the list goes on. Permissive.

What about the data stores? Server-side code typically connects with those through client/driver libraries of some sort. PostgreSQL, CouchDB, Redis, Riak, Cassandra, Memcached, SQLite – all permissive licenses. MongoDB is AGPL but the client is permissive (Apache) so there’s no copyleft affecting your program that talks to the DB. MySQL is more complex. After all it was the product that defined dual-licensing so that’s not entirely unexpected. The DB itself is GPL, but the client library is LGPL in really old versions, GPL with a FLOSS exception in versions prior to 5.1.55 and plain GPL in recent versions released by Oracle. Oracle also provides a PHP client library under the PHP license. You can also acquire a commercial license of MySQL, with a non-copyleft client library. So whether you can write a proprietary program that talks to MySQL, depends. That’s a whole lot of babbling about data stores licenses, pardon.

Not everything is permissive of course. A good amount of projects use limited-copyleft licenses such as the MPL, EPL and LGPL. The intent is almost always to allow for proprietary usage of the library/framework/program, but to mandate that modifications and additions to the project itself get contributed back, to various extents. EPL is particularly common in Java-camps (so is the permissive Apache 2.0 mentioned earlier). Mozilla use MPL for most (all?) of their stuff, and it also happens to be the most permissive of the three.

And then we have the regular copyleft licenses, most notably the GPL, which isn’t that popular when it comes to web development libraries and frameworks. Sencha’s ExtJS framework is GPL/commercial dual-licensed. Many avoid it for that reason, and for those there are similar projects available under permissive licenses. Wordpress is a another GPL project that is very popular.

If we start looking under the hood, there’s a good amount of permissive licensing going on in other parts of the web stack. PHP, Ruby, Python, Apache HTTPD and nginx are all permissive. There’s also a good amount of GPL and LGPL, most notably the Linux kernel and the GNU userland.

Let’s focus again on client- and server side web development libraries/frameworks. By far and large, the web developer community seems to prefer permissive licenses for that. Permissive licenses are proprietary-friendly, and many web developers write proprietary programs for a living. Many also don’t have a problem with giving out their code for others to use however they wish, save for some attribution. Most importantly, many consider the idea of proprietary programs and services to be fine, sound, ethical and all that. It’s also fairly obvious that permissive OSS projects can attract huge contributions even without any help from a license that mandates it. Imagine Node.js or Rails being GPL. Would they have received more or less contributions, you think?

I hope you won’t blame me for calling web developers “rewrite-happy”. Heck, the entire programmer population is, to some extent. Someone takes the ideas of a library or framework and rewrites it in another language. Someone else rewrites it from scratch just because he/she (hen!) was unhappy with a particular design decision. Competing frameworks overlap. Just look at the throne of JS conference. Now this is both good and bad. Good in the sense that it’s very evolutionary and that it constantly pushes innovation forward. Bad when yet another author loses yet another potential contributor to yet another similar-looking 0-day friendly competitor. But it’s mostly good.

The publicly available Meteor code base is licensed as GPL (v2). Any application you write using it - definitively the client part and quite likely the server part as well (due to Meteor’s tightly intertwined client and server architecture) - any such application must be released under GPL compatible terms. I.e. you must hand out the source code, and no you can’t obfuscate it. If you want to write a proprietary program then you need to have a discussion with the Meteor team about how to acquire a commercial license. Combining rewrite-happy with a preference for permissive licenses, I think it’s reasonable to expect existing project to adapt a few of Meteor’s core features and design ideas, and new ones popping up as friendly competitors. Hardly shocking.

The thing about Meteor is that they really want to establish an ecosystem of community-contributed smart packages, harnessing the goodness of existing Node.js and JavaScript server- and client side libraries. Those smart packages must be GPL or GPL compatible (most permissive licenses are). For Meteor to be able to include a contributed smart package with their commercial offering, the contributed smart package must either use a permissive license or come with a separate contributor agreement that allows Meteor to ship it commercially. Otherwise the GPL backfires in the sense that your contributed GPL smart package cannot be combined with a proprietary version of Meteor.

The copyleft (viral, contaminating, whatever you want to call it) aspect of GPL is tricky. Take MongoDB as an example. Meteor uses it by importing the node-mongodb-native package (require('mongodb')). That one is Apache 2.0 licensed, which is a permissive license that happens not to be compatible with Meteor’s GPL (v2) license, at least not according to the FSF. Tricky. Dependency chains, bindings between JS ↔ C and RPC makes it trickier even. I wouldn’t be surprised to see Meteor change to a GPL + a-bunch-of-OSS-exceptions license similar to what Qt and MySQL used to have, to avoid issues like this.

I’m curious to find out to what extent the Meteor ecosystem will attract outside contributors, and to what extent people will start using the GPL version of Meteor in their projects (accepting the copyleft). Copyleft code grows from one project to another because it must or else the license screams VIOLATION. Permissive code grows from one project to another by culture and an honor system. It’s like, I used all this code and all that was required by me was a tiny bit of attribution (if that even) - I wasn’t limited in terms of how to combine it with other code. When I choose to pitch in something in return, it’s only fair of me to be permissive too. That’s how I perceive the mindset of the web development community at large.

And yeah, I used a sensational title. Tricky.

Update 2012-04-21: Meteor has switched to the permissive MIT license.

Show comments (Hacker News)

Speaking at DevLin and Front-Trends

2012-02-14T22:00:00+01:00

I’m excited to be invited to speak at DevLin2012 (Sweden) and Front-Trends (Poland). Lots of interesting topics there that I want to attend. Summaries of my own talks follows below:

Evolution vs. revolution with JavaScript and Dart (DevLin2012)

There are clear benefits to developing modern web apps but what are the drawbacks? How are we limited by the programming language JavaScript which in practice means ECMAScript 3 from 1999?

We’ll cover different opportunities and challenges such as performance, startup time, scalability with a growing code base and developer team, productive development environments, strict enough but at the same time expressive enough language semantics, tools for code analysis and more.

The upcoming ECMAScript 6 represents an evolution of today’s JavaScript. Dart starts from scratch with a newly designed language for the web. How do they relate to the challenges above? When can you start using them and what do they require from the end-user environments?

How can we make the best of JavaScript as we know it today?

JavaScript: the subsets we use (Front-Trends)

Is it possible to make our programs more robust, easier to reason about and more enjoyable to write by carefully restricting our usage of the JavaScript language? Let’s talk about that.

Olov Lassus is a Swedish passionate programmer who started his career in the 16-bit game development era of the early 90′s. He got into JavaScript by accident when writing his master’s thesis and it has pursued him ever since. He likes to fiddle with language design, programmer productivity and tooling.

Pygments contribution now upstream

2012-02-05T12:30:00+01:00

My Pygments Dart highlighter that I’ve written about previously is now upstream! This means that there’s now out of the box support for Dart if you get the development version from source control and that the next formal release is likely to have it too. Github enabling it shouldn’t be far away now.

Show comments (Dart misc)

Dart syntax highlighting update

2011-11-26T15:00:00+01:00

Here’s a short update on the Pygments Dart support I created a month back.

I consider it good to go now and will submit it to the Pygments upstream and github shortly. The most notable addition is support for string interpolation. I’ve also created a custom color scheme (based on tango) that works well with at least Dart, JS and Diff.

It looks like this:

#!/bin/dart

main() {
  String name = "Zoidberg";
  const age = 42;
  var liar = Math.random() > 0.5;

  // Present ourselves
  print("Hi! My name is $name and I'm ${liar ? age / 2 : age} years old");
}

Source code is available at https://gist.github.com/1320163. Install it by appending the gist to Pygments’s agile.py and add the following line to _mapping.py.

    'DartLexer': ('pygments.lexers.agile', 'Dart', ('dart'), ('*.dart',), ('text/x-dart')),

Then run pygmentize -f html -o yourfile.html yourfile.dart to generate markup from your source code. You’ll need to combine it with a style sheet. Feel free to copy my color scheme (syntax.css) if you like it.

Show comments (Dart misc)

Let's create us some Dart ranges

2011-11-21T11:27:00+01:00

We recognize the good ol’ for loop from the C-family of programming languages. It is available with the same familiar syntax and semantics in Dart, too:

for (int i = 0; i < 10; i++) {
  print(i);
}

In addition to that, Dart has a for-in loop, similar to Java in semantics (iterator(), hasNext() and next()). for-in is used to conveniently loop over any Iterable sequence, such as a List.

for (int i in [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]) {
  print(i);
}

If you’re into Python then you know that over there you for loop over sequences exclusively, and use xrange extensively.

for i in xrange(0, 10):
    print(i)

xrange is quite convenient and prevents a lot of variable name stuttering. It takes up to three parameters: start, stop and step. Python xranges exclude the stop endpoint and step is 1 by default. start is 0 by default so you’ll often see xrange(len(seq)) used in Python code to iterate over sequence indices.

Back to Dart. While good ol’ for loops have been working fine for the last few decades or so, perhaps something xrange-like could be fun here too. We’ve already got for-in and iterators after all. Let’s create a Range class that implements the Iterable interface with a corresponding RangeIterator and we’ll be able to do for-in range, with top-level function range being a wrapper around the Range constructor. We’ll include both endpoints so range(0, 9) means [0, 1, .. 9]. Again step defaults to 1 so range(0, 9) is the same as range(0, 9, 1).

for (int i in range(0, 9)) {
  print(i);
}

That looks swell. No surprise it has already been discussed on dart-misc. We can also pass a range to anything that expects an Iterable. Here’s how to generate the List [0, 2, .. 8] using the List.from constructor:

var list = new List<int>.from(range(0, 8, 2));

Iterating over indices is done like so:

var str = "hellojed";
for (int i in range(0, str.length - 1)) {
  print("$i: ${str[i]}");
}

We’ll add a top-level function indices to sweeten it a bit:

for (int i in indices(str)) {
  print("$i: ${str[i]}");
}

We can make range more useful still. The Collection interface (implemented by List and Set among others) contains length, every, filter, forEach, isEmpty and some. Let’s implement those too. Now we can pass around ranges anywhere a Collection is expected, for example to addAll:

var queue = new Queue<int>();
// ...
queue.addAll(range(0, 9));

How about a List with the numbers [0, 1, .. 50] that are perfect squares:

bool isSquare(int n) => Math.pow(Math.sqrt(n).round(), 2) == n;
var squares = range(0, 50).filter(isSquare);

Or the other way around? map isn’t part of Collection but let’s add it anyways:

var squares = range(0, 7).map((e) => e * e);

The original example can also be rewritten like this, using forEach:

range(0, 9).forEach(print);

Not too shabby, all in all. Good ol’ for loops have less overhead than their for-in range equivalent but the difference is nothing to write home about, especially if your loop body does something useful. Your personal style is probably more important - as for myself I consider range a nice complement.

Check out the tests for more examples. The implementation should be easy to follow if you want to learn how to create a Collection of your own. If you want a little Dart exercise then change Range so that it only implements Iterable<int> and remove everything Collection-specific. Then add it back again guided by the unit tests, this time with your own code. Or start from scratch altogether.

Oh and the Dart for-in loop is just (very convenient) syntactic sugar of something similar to this:

for (Iterator it = range(0, 9).iterator(); it.hasNext(); ) {
  int i = it.next();
  print(i);
}

dart-range source available on github.

Show comments (Dart misc)

Dart syntax highlighting support for Pygments

2011-10-27T19:35:00+02:00

Source code is available at https://gist.github.com/1320163. Install it by appending the gist to Pygments’s agile.py and add the following line to _mapping.py.

    'DartLexer': ('pygments.lexers.agile', 'Dart', ('dart'), ('*.dart',), ('text/x-dart')),

Dart syntax highlighting example:

// Greeter example from
// <http://www.dartlang.org/docs/getting-started/interface.html>
class Greeter implements Comparable {
  String prefix = 'Hello,';
  Greeter() {}
  Greeter.withPrefix(this.prefix);
  greet(String name) => print('$prefix $name');

  int compareTo(Greeter other) => prefix.compareTo(other.prefix);
}

void main() {
  Greeter greeter = new Greeter();
  Greeter greeter2 = new Greeter.withPrefix('Hi,');

  num result = greeter2.compareTo(greeter);
  if (result == 0) {
    greeter2.greet('you are the same.');
  } else {
    greeter2.greet('you are different.');
  }
}

Show comments (Dart misc)

Describing Live programming (again)

2011-10-09T17:00:00+02:00

It’s funny how the brain does context switching sometimes. You work really hard on something just to almost forget about it immediately after the release. You then let it be for a few years until you dare to open up that project or document again to figure out why you worked that hard on it in the first place.

Anyways. I wrote my master’s thesis in 2006 and I was really passionate about it at the time. I wanted to describe how to create a better programming environment than what we currently had without falling into the “everything has already been invented in good ol’ Smalltalk and the LISP machines” trap. I specifically focused on enabling the programmer to write and modify code at run-time, thus the name Live programming.

I was satisfied with what I accomplished. I had a prototype running that I used to test that the source code, AST and callstack transformations I described in the thesis were valid. I knew what the next step would be (post thesis) but I also knew that I didn’t have the time to invest into making a full blown implementation, so I let it be.

Almost six years has passed since then and the programming environments people use today haven’t changed much. I’m still passionate about Live programming. I figured it wouldn’t hurt to expose the concept to a new reader or two, thus this blog article.

When I wrote the thesis I was primarily interested in finding out how far I could take the concept. I picked JavaScript as the prototype language because it seemed to have momentum at the time and I guess that choice turned out well. I picked Narcissus because it was the simplest interpreter I could find. The prototype environment uses a heavily modified Narcissus (I turned it stackless/callstack-explicit) and it seems to have bitrot since then but I’ll see if I can beat the code into shape and release it out again if there’s any interest.

I still think that JavaScript is an excellent language for implementing a Live programming environment. I’d love to see one integrated directly in the browser, and also tied together with a node.js environment. It needs to be directly integrated with the JS impl, i.e. the V8, SpiderMonkey or what have you. It’s not a small feat.

So here we go: Describing Live programming using program transformations and a callstack explicit interpreter. Read chapter 1 and 2 for more info about Live programming. Read the other chapters only if you’re interested in how it works under the hood (the transformations).

Oh and I did an s/Johansson/Lassus thing in 2007, in case you wonder.

Show comments (reddit)

Seems like restrict mode found us a jQuery bug

2011-06-16T19:30:00+02:00

Feel free to skip the first paragraph if you’ve read my previous restrict mode articles.

I’ve created this thing called restrict mode for JavaScript. It’s a definition of a proper subset of the language which I believe is great because I think it increases robustness and makes your program more easy to reason about. It comes with a tool that will check if your program really complies to restrict mode, when you say it does. The tool is called restricter and is a plugin for JSShaper, an extensible framework for JavaScript syntax tree shaping. Restrict mode makes JS programming more enjoyable since it helps you focus on real programming challenges instead of annoyances/defects/inconsistencies of the JS language.

I’ve written about how I made v8bench and JSLint restrict mode clean and I promised to do the same for jQuery.

I’ve tweaked the restrict mode semantics somewhat since then:

I removed the restrict mode custom versions of == and !=. This shouldn’t come as a big surprise since I hinted that I wasn’t happy with clause 1 in the case for restrict mode. Now all of ==, !=, === and !== behave identically in normal and restrict mode and you should strongly consider not using == and != irrespective of whether you use restrict mode or not.
I relaxed the restrict mode + operator slightly. It was previously only allowed for str + str (concatenation) or num + num (addition), but never a mix. It’s now also allowed for num + str or str + num, which happens to be the most common coercing usage of it out there in the wild. I hope that this will lower the barrier of restrict mode adoption.

Restrict mode is still a proper subset of the JavaScript language and will remain so. It will freeze one day but we’re not there yet.

Back to jQuery. I cloned the jQuery repo, ran it through restricter (with restrict mode forced no matter "use restrict"; directives) and fixed the restrict mode run-time errors until all tests passed again. restricter ran on everything under src/ excluding the Sizzle dependency, to be precise. It should be easy to try it on Sizzle as well, or on something else for that matter.

The required changes to get jQuery restrict mode clean were few and small. They can be partitioned into three classes:

Most were about fixing the str + nostr_nonum antipattern
One fixing str < num
One fixing str + undefined

Neither class 1 or 2 are bugs in jQuery. They represent the current mismatch between the restrict mode subset of JavaScript and the subset that the jQuery developers use. The mismatch seems to be very small, further strengthening the case for restrict mode. I only had to change a total of 9 lines of code, all by explicitly converting a primitive or object into a string using String(expr). The patched version may be a bit easier to reason about and slightly more robust to future changes, especially so for newcomers to the jQuery codebase. Being explicit makes it slightly more verbose.

The class 3 change is more interesting. str + undefined is rarely done on purpose. jQuery.fn.hasClass should guard against missing className properties but doesn’t, and as a consequence element.hasClass("undefined") may return true when it shouldn’t. An easy way to demonstrate this is to change test/unit/attributes.js, line 860. Change ok(!j.hasClass("asdf"), ...) to ok(!j.hasClass("undefined"), ...) and watch the assert blow. Talking about assert feel free to check out C-style assertions in JavaScript via JSShaper.

Seems like restrict mode found us a jQuery bug.

Here’s the patch aggregating the changes above including the bugfix. It applies cleanly against d59b0f3e27827d189b8b2595142ec6bbc3941dd9. And here’s the jQuery forum thread, ticket and pull request.

Want to try restrict mode on your own project? Go to restrictmode.org and feel free to stop by the JSShaper mailing list.

Show comments (reddit)

I made JSLint restrict mode clean. Here's what I found

2011-04-21T20:50:00+02:00

I’ve created this thing called restrict mode for JavaScript. It’s a definition of a proper subset of the language which I believe is great because I think it increases robustness and makes your program more easy to reason about. It comes with a tool that will check if your program really complies to restrict mode, when you say it does. The tool is called restricter and is a plugin for JSShaper, an extensible framework for JavaScript syntax tree shaping. JSShaper is cool for many other reasons as well but this post is about restrict mode. For me restrict mode makes JS programming more enjoyable since it helps me focus on real programming challenges instead of annoyances/defects/inconsistencies of the JS language. It eases my mind.

I want to know how well the restrict mode subset of JavaScript matches existing programs. I’ve already written about how I made v8bench restrict mode clean. I’ve also done the same to jQuery and Kraken, though I haven’t written about it yet (I’m worse at writing than coding but it’s on my todo-list). When I saw a JSLint discussion on reddit I figured that it would be a great test case, so I decided to make it restrict mode clean and write about what I found. I’m glad I did because it exposed a bug in JSShaper (now fixed). I also think that it serves as a good example of what you can expect from restrict mode.

Here are my findings and how I changed JSLint to make it restrict mode clean. In my checkout of JSLint, commit 99091cd30d354e77cf95 was the latest and greatest so the changes apply cleanly against that revision.

First of all, we need to specify which parts of the program should follow restrict mode. We want it to cover all of JSLint so that the JSShaper restricter plugin can help us catch any issues. JSLint is already enclosed in a function which starts with a "use strict" string. Let’s add "use restrict" as well:

var JSLINT = (function () {
-    "use strict";
+    "use strict"; "use restrict";

To show what I changed I’ve included the diff’s (ceremony removed). Red - lines are what I removed, green + lines are what I added. Scroll down for full patch.

With that change in place, I ran jslint.js through JSShaper (restricter): js run-restricter.js -- JSLint/jslint.js > JSLint/jslint_restrict.js. I tweaked the jslint.html file to use the restrict mode checked version and to dump the errors to console instead of throwing exceptions. To make JSLint restrict mode clean I just had to inspect the errors (when running JSLint) and fix the issues. I iterated on this, and here’s what I found and changed:

First change is to fix a coercion-happy String.prototype.isAlpha. When "hello".isAlpha() or str.isAlpha() is evaluated the expression left of the dot is first boxed into a temporary String object, effectively the same as (new String("hello")).isAlpha() or (new String(str)).isAlpha(). In the String.prototype.isAlpha function, the temporary object is bound to this. [This behavior was then changed in ES5 strict mode to further add to the confusion since it’s not backwards compatible, which is another reason for First change]. If this is used as an operand like below it will trigger coercion, sometimes with unexpected results. You can get the primitive value from a String object by applying the valueOf method or String function. Let’s do that, so that the <= and >= operators won’t need to coerce their operands any longer:

String.prototype.isAlpha = function() {
-            return (this >= 'a' && this <= 'z\uffff') ||
-                (this >= 'A' && this <= 'Z\uffff');
+            var v = this.valueOf();
+            return (v >= 'a' && v <= 'z\uffff') ||
+                (v >= 'A' && v <= 'Z\uffff');
         };

Second change is to do the same for String.prototype.isDigit:

String.prototype.isDigit = function () {
-            return (this >= '0' && this <= '9');
+            var v = this.valueOf();
+            return (v >= '0' && v <= '9');
         };

Third change is to fix String.prototype.name. It used to return this sometimes, which is not the right thing to do. I described above how primitives get boxed into temporary objects. Returning these objects is almost always bad for two reasons: It spreads the coercion disease, and it’s not what the user of the function expects (because now sometimes the function returns strings, sometimes it returns String objects). Let’s fix that by returning this.valueOf(). Let’s also use the primitive value with the + operator, and the coercion is gone. Here we go:

String.prototype.name = function () {
-            if (ix.test(this)) {
-                return this;
+            var v = this.valueOf();
+            if (ix.test(v)) {
+                return v;
             }
-            if (nx.test(this)) {
-                return '"' + this.replace(nxg, function (a) {
+            if (nx.test(v)) {
+                return '"' + v.replace(nxg, function (a) { // this.replace works too
                     if (escapes[a]) {
                         return escapes[a];
                     }
                     return '\\u' + ('0000' + a.charCodeAt().toString(16)).slice(-4);
                 }) + '"';
             }
-            return '"' + this + '"';
+            return '"' + v + '"';
         };

Fourth and final change is to fix the report function which performs a lot of string formatting relying on numbers being coerced into strings (str + nostr). It’s a fine example of code that would greatly benefit from using Fmt or something similar. There’s also the option to bail out of restrict mode for the entire report function by annotating it with /*@loose*/. For this example I fixed it in the least obtrusive manner by applying the String function to the numbers. Here we go:

@@ -6523,7 +6526,7 @@ loop:   for (;;) {
-                            warning.line + ' character ' + warning.character : '') +
+                            String(warning.line) + ' character ' + String(warning.character) : 
@@ -6536,7 +6539,7 @@ loop:   for (;;) {
-                        data.implieds[i].line + '</i>';
+                        String(data.implieds[i].line) + '</i>';
@@ -6545,7 +6548,7 @@ loop:   for (;;) {
-                        data.unused[i].line + ' </i> <small>' +
+                        String(data.unused[i].line) + ' </i> <small>' +
@@ -6583,7 +6586,7 @@ loop:   for (;;) {
-                output.push('<br><div class=function><i>' + the_function.line + '</i> ' +
+                output.push('<br><div class=function><i>' + String(the_function.line) + '</i> ' +

And that’s it. JSLint is now restrict mode clean, as far as I can tell. I didn’t find any tests but I’ve tested it manually with a few programs, including JSLint itself.

What have we learned?

For a start, it’s another step in the right direction in verifying that the restrict mode subset of JavaScript is sane and well enough matches the JS subset people already use in practice. It took little effort to get JSLint restrict mode clean, and it required no prior experience with that code base. We had to change a few lines of code as expected, and the signal vs noise was very high.

Did we find any real defects? We caught String.prototype.name sometimes returning temporary boxed objects which I classify as a defect, even though it doesn’t cause any harm at the moment. Fixing that is a keeper.

Even if we don’t buy into the idea restrict mode, is there something else that we’d like to keep? I’d suggest the fixes for coercion happiness examplified in change two and three. Especially so if you follow the recommendation of preferring === over == (I do). Think about it: It would be a bit weird to see this >= '0' && this <= '0' yield true when v is a temporary boxed String object (it will coerce), but have this === '0' yield false (it won’t coerce). Leaving it as it is will increase the risk of future bugs. If you don’t prefer === over == then you’re likely not to care about this change.

And some of you would perhaps consider keeping the changes for our fourth finding as well (str + nostr) even if you didn’t yet buy in on restrict mode, because quite frankly the string formatting code in that function is not the most beautiful or easiest code to follow. Using Fmt or something similar could greatly improve it. Many will consider the code fine as-is and won’t change it unless they go restrict.

To summarize: Of our four findings/changes, three seem noteworthy enough that I think many would consider applying the changes even if they don’t care about restrict mode. Not bad for a few hours of work of which the majority went into writing this post.

If you choose to keep all changes as well as the "use restrict" directive, then you can easily run the program through JSShaper (restricter) again in the future and perhaps find more stuff. It could help you during development, and it could help your program become more robust. You can choose to run the checked version only once in a while, every time your unit tests run or all the time. It’s totally up to you.

A program complying to the subset of JavaScript defined by restrict mode (i.e. a restrict mode clean program) is also much easier to reason about compared to a program that (may) use the entire spectra of loose JS semantics. That may help you when debugging tomorrow or when coming back to your old code one year from now. If you read your friend’s code and happen to find the "use restrict” directive in there, you may find yourself relieved knowing that not only will you faster and more accurately understand what it does and how, chances are that your friend accurately understands his/her own program too.

The JSLint restrict mode experiment exceeded my expectations. Why don’t you try applying restrict mode to one of your own programs? What’s to lose?

There’s more info available on restrictmode.org. If you have any questions about restrict mode or JSShaper then post a message to the JSShaper mailing list and I’ll help you out. You can also contact me directly.

Here’s the patch for everything described above (jslint.html and jslint.js).

Show comments (reddit)

JavaScript modules in shells, browsers, everywhere!

2011-04-04T22:52:00+02:00

This is the technique I’m using to get JavaScript module support in the console and in the browser, with the same code base. It’s tested with the following JS shells: js (SpiderMonkey), jsc (JavaScriptCore), v8/d8 and node. It’s useful since it lets you stay in the console as much as you want to and it enables you write once, run everywhere (that’s what they said).

I’m using it in JSShaper. Do you find it useful? Do you know of a better way? Let me know!

In your program top-level file, define require as follows. It’s now either defined as the built-in require or a version of the built-in load that returns nothing.

var require = require || function(f) { load(f); };

In all other files, when you need to import a module such as Fmt, do it like this. The first || clause is truthy (and thus assigned) when Fmt is already defined in the global namespace, the second when require is CommonJS-style and the third when require wraps load (thus Fmt is now in the global namespace).

var Fmt = Fmt || require("./fmt.js") || Fmt;

Finally, your modules should have the following structure, for example Fmt:

var Fmt = (function() {
    // ...
    return ...;
})();
if (typeof exports !== "undefined") {
    module.exports = Fmt;
}

Voila, your module-separated program now works in the JS shells and the browsers! For the browsers, just include (or concatenate) the scripts in dependency-order. This isn’t a dynamic JS loader - if you want one there’s gazillions of them.

Here are the underpinnings for the curious:

In node, require is a built-in function used to load CommonJS-style modules. require reads code from disk and evaluates it. It returns the object exported by the module. The module lives in its own namespace. There’s no load function.
In v8, load reads code from disk and evaluates it in the global namespace. It returns nothing (undefined). There’s no require function.
In js, load reads code from disk and evaluates it in the global namespace. It returns the load function itself (don’t ask me why). There’s no require function.
In jsc, load reads code from disk and evaluates it in the global namespace. It returns the value of the last statement in the file (or something like it). There’s no require function.
In browsers, there are no load or require functions.

Show comments (reddit)

Making v8bench restrict mode clean

2011-04-01T00:37:00+02:00

I’d like to know how well the restrict mode subset of JavaScript matches existing programs. I decided to test it on Google’s v8bench. I checked out the latest revision from SVN, and slapped "use restrict"; on top of every file. I ran that through restricter (part of JSShaper) before executing v8bench.

The benchmark then threw an exception whenever restrict mode detected that things were going on outside of its rules. I tweaked the v8bench source, and iterated.

I had to change only 8 lines of code to get v8bench restrict mode clean, i.e. have it complete without throwing any restrict mode exceptions. That’s 8 lines out of 11208, or 0.07%! 99.93% was already valid restrict mode. It’s a step in the right direction in verifying that the restrict mode subset of JavaScript is sane and well enough matches the JS subset people already use in practice.

Oh, and if the same procedure is applied to r2537 instead of HEAD, one defect in the splay benchmark (converting the same numeric key to a string over and over again) stands out and is easily corrected. That defect was later fixed in version 6 of the benchmark. Can restrict mode help you catch similar or other defects in your program?

To learn more about restrict mode read the case for restrict mode and then visit restrictmode.org.

Here’s the full diff:

Index: run.js
===================================================================
--- run.js (revision 7463)
+++ run.js (working copy)
@@ -38,7 +38,7 @@
 var success = true;
  
 function PrintResult(name, result) {
-  print(name + ': ' + result);
+  print(name + ': ' + String(result));
 }
  
  
Index: raytrace.js
===================================================================
--- raytrace.js (revision 7463)
+++ raytrace.js (working copy)
@@ -617,8 +617,8 @@
                 rayDepth: 2
             }, options || {});
  
-        this.options.canvasHeight /= this.options.pixelHeight;
-        this.options.canvasWidth /= this.options.pixelWidth;
+        this.options.canvasHeight /= Number(this.options.pixelHeight);
+        this.options.canvasWidth /= Number(this.options.pixelWidth);
  
         /* TODO: dynamically include other scripts */
     },
Index: base.js
===================================================================
--- base.js (revision 7463)
+++ base.js (working copy)
@@ -201,10 +201,10 @@
 BenchmarkSuite.prototype.RunSingleBenchmark = function(benchmark, data) {
   function Measure(data) {
     var elapsed = 0;
-    var start = new Date();
+    var start = Number(new Date());
     for (var n = 0; elapsed < 1000; n++) {
       benchmark.run();
-      elapsed = new Date() - start;
+      elapsed = Number(new Date()) - start;
     }
     if (data != null) {
       data.runs += n;
Index: deltablue.js
===================================================================
--- deltablue.js (revision 7463)
+++ deltablue.js (working copy)
@@ -800,7 +800,7 @@
  
   // Build chain of n equality constraints
   for (var i = 0; i <= n; i++) {
-    var name = "v" + i;
+    var name = "v" + String(i);
     var v = new Variable(name);
     if (prev != null)
       new EqualityConstraint(prev, v, Strength.REQUIRED);
@@ -836,8 +836,8 @@
  
   var dests = new OrderedCollection();
   for (var i = 0; i < n; i++) {
-    src = new Variable("src" + i, i);
-    dst = new Variable("dst" + i, i);
+    src = new Variable("src" + String(i), i);
+    dst = new Variable("dst" + String(i), i);
     dests.add(dst);
     new StayConstraint(src, Strength.NORMAL);
     new ScaleConstraint(src, scale, offset, dst, Strength.REQUIRED);

Show comments (reddit)

C-style assertions in JavaScript via JSShaper

2011-03-30T00:35:00+02:00

Earlier today I created asserter, a new plugin for the JSShaper framework that provides you C-style assertions in JavaScript.

Here’s a common style of doing JS assertions:

function myfn(x) {
    assertTrue(x >= 0, "argument x must be positive or zero");
}

When failing (let’s say x is -1) it will print something like this, perhaps with some kind of cleaned up stack trace:

assertTrue() failed: argument x must be positive or zero

This is what you would write instead when using asserter:

function myfn(x) {
    Assert(x >= 0);
}

When failing, it will print this:

Assertion failed: x >= 0, function myfn, file prog.js, line 2

I know which version I prefer. Thanks to the JSShaper framework I was able to create the asserter plugin in no time, using only 34 29 (edit: shorter and easier with declarative matching) lines of code.

Here’s how it works. asserter searches the syntax tree for calls to the Assert function. It adds an extra string argument containing the assert condition expression (in text form), the enclosing function name (or <anonymous> or <script>), the filename and lineno. Here’s what it would be replaced with:

Assert(x >= 0, "x >= 0, function myfn, file prog.js, line 2");

The Assert function could look like this. Note that it will execute just fine even if you didn’t run your program through asserter, although in that case it will just print a falsy value.

function Assert(condition, str) {
    if (!condition) {
        log("Assertion failed: "+ (str || String(condition)));
        quit(-1);
    }
}

Asserting preconditions in JS has never been simpler, no?

In C you would disable assertions in your release-build (allowed since assertions mustn’t contain side effects). In JS we could do the same by simply creating another JSShaper pass that removes the Assert calls.

Show comments (reddit)

The case for restrict mode

2011-03-28T21:01:00+02:00

I sent this mail to the JSMentors list yesterday [full thread]:

I’m announcing restrict mode for JavaScript, a proper subset of the language designed for improved program robustness. It’s opt-in similar to ES5 strict mode and enabled with a string literal ("use restrict";). Continue reading for an explanation of how it works and why I think it’s useful.

Operator semantics in regular (I call it loose) JavaScript are complex. In many cases the increased capabilities of this complexity doesn’t really outweigh the drawbacks, which include less robust programs and a language that’s both harder to learn for the beginners and harder to reason about for the experts.

Of course, this is nothing new. The idea of writing programs that for example perform advanced number arithmetics on operands of type strings seems silly. Still, the language lets us do exactly that. I’m arguing that much of the operator semantics complexity in the language, with corresponding type coercion, is not wished for. If we could make some of it go away, fewer bugs would hide and our programs would become more robust. It may also become even more fun to write and debug them.

This was discovered years ago when people started recommending the use of the (strict) === operator instead of the (loose) == operator, and the same for !== vs !=. It’s entirely possible to write programs without using the == operator at all, and many programmers have chosen to do so. The result? More robust programs that are easier to reason about. For a beginner, it’s also a programming style that’s easier to learn and master.

The problem is that there is no strict version of the (loose) < operator. Or >, <= , >=, + or - and so on and so forth. Because these are loose and loose-only, we’ve kind of accepted them as-is and moved on. Imagine for a second what would happen if you suddenly had strict versions of these operators available, with the possibility of making the strict versions the default for parts of your program. How would you then define the strict version of <? I bet you wouldn’t want it to accept an operand of primitive type number and another of primitive type string, since that would most likely be a bug in your program. Would you want it to accept undefined < 3? I bet not. Perhaps you would like to get notified as early as possible with an exception if your strict rules are broken?

Here’s the thing. We already limit ourselves to a subset of the semantics for these operators. At least we try to. We don’t mix strings, numbers and booleans randomly. We don’t perform arithmetics to undefined, on purpose. We try to be in control of when type conversion happens, and so on. It’s just that we don’t necessarily use precisely the same subset, because we haven’t really though about what we should be restricting our programs to. And we don’t have any good means of getting notified as soon as we break that subset. We would like to, because no matter how good we are at this language, we still have to fix defects in our code. Defects some of which are related to breaking the subset.

As you might have guessed by now I’ve defined such a subset and created a tool that modifies a JavaScript program so that it throws an exception whenever the subset is broken. I call the subset restrict mode for JavaScript and would like to get your feedback on it. It’s opt-in similar to ES5 strict mode, and you can easily opt-out for any part of your program. The tool is called restricter. It’s a plugin for JSShaper, an extensible framework for JavaScript syntax tree shaping.

The restricter-modified program should be used when running your test suite. Since restrict mode is a proper subset of regular JavaScript, you can still deploy the original program.

Here are the restrict mode rules:

0. === and !==
   are already strict by default, with identical semantics in
   restrict mode
1. == and !=
   are restricted to undefined == null (true) and
   undefined != null (false). They are otherwise only allowed
   when the === or !=== operator yields an identical result
2. + < <= >= >
   are restricted to primitive strings or numbers
   (but never a mix)
3. - * / % & | ^ ~v << >> >>> -v ++v v++ --v v--
   are restricted to primitive numbers
4. That's it!
   +v !v && || ?: o[k] and all other operators
   have identical semantics in restrict mode

Many programs already conform to most of these rules, with the notable exception of the restrict + operator in clause 2. In restrict mode you can no longer use str + nostr to perform string conversion and concatenation at the same time. You can easily replace this with a call to a string-formatting function (I use Fmt) or an explicit conversion via String(nostr).

Clause 1 is a compromise and may need to change. I’d rather see == and != disallowed but there seem to be many programs relying on the undefined == null behavior. For now I’ve added a pedantic setting where it’s disallowed.

I’d like to hear your thoughts on all this. Does restrict mode resonate with you and if so, do the rules define the right subset?

I have a preliminary web site up at http://restrictmode.org/ with more information. You can try restrict mode live in your browser by following the “Try restrict mode” link. Source code for JSShaper is available at https://github.com/olov/jsshaper.

/Olov

Show comments (reddit)

Announcing restrict mode for JavaScript

2011-03-27T00:34:00+01:00

Welcome redditors! I’m answering your questions in the comment thread.

"use restrict";

Add that to the top of your JavaScript program. Run it through the JSShaper restricter plugin, then combine it with the restrictmode.js prelude and run it. Then hook it up with your units tests.

Try restrict mode directly in your browser (Firefox, Chrome or Safari)!

For semantics and rationale visit restrictmode.org. It’s not set in stone so let me know what you think.

I’ll write more about JSShaper in time. It’s an extensible framework for JavaScript syntax tree shaping and we’re going to be able to do lots of nifty things with it.

Show comments (reddit)