State of RapydScript

Matrix-agent-Smith-clones Many of you probably noticed that I’ve been slow to contribute to RapydScript lately. It’s hard to summarize all of the reasons, but here are the top few:

  • Despite the occasional bug, the project is in solid state and quite usable in production (I do eat my own dog food by the way).
  • I’ve been working on other projects, most recent one being TileZone.
  • I am one man, and my time doesn’t scale. I wish I could continue improving the language and fix the bugs people report but at the end of the day I don’t get paid for it, and as a result I prioritize my other projects and my job over this. I’m grateful for the contributions from other members, but the community isn’t large enough to encourage others to contribute regularly.
  • With ECMAScript 6 and Babel, the future of RapydScript seems somewhat uncertain. Babel is an awesome library that brings many similar features to RapydScript (although with uglier syntax, in my opinion). I think, however, that RapydScript, if continued, will be more relevant in ES6 world than something like CoffeeScript.
  • As mentioned before, I don’t believe Python is a panacea. It’s one of my favorite languages, but I’ll be the first one to admit that it made quite a few mistakes design-wise. Some issues that come to mind are global-scope collection methods (len, map, filter), lambda functions, inconsistent naming scheme for native objects (why are native classes called int and str when the convention is to use title case for them?), exposing of optimization methods (xrange) to the user when they should really be handled behind the scenes, and so on. As a result, it’s frustrating to see new RapydScript users complain that the language should be more like Python, especially the parts of Python that I hate.

With that said, I love RapydScript, and will continue using it. I prefer it to both, Python and JavaScript. Ironically, I started RapydScript to avoid dealing with JavaScript, but in the process learned to appreciate it more. Some design patterns I use in RapydScript would not be possible in Python. JavaScript is a powerful language, and yet it’s messy. This StackExchange answer to a question about language verbosity does an amazing job explaining what I like about RapydScript. If we were to grade the 4 languages I mentioned on those 2 categories, I would rate them as follows:

Language Signal vs Noise Cryptic vs Clear
JavaScript ⭐⭐⭐⭐⭐
CoffeeScript ⭐⭐⭐⭐⭐
Python ⭐⭐⭐⭐ ⭐⭐⭐⭐
RapydScript ⭐⭐⭐⭐ ⭐⭐⭐⭐⭐

JavaScript is definitely the most verbose of the 4, having to type out all of those prototypes, optional argument checks, array[array.length-1], and brackets gets repetitive very fast. Looking at all that code gets repetitive even faster. As far as CoffeeScript, there is no arguing that it’s terse. Its signal to noise ratio is amazing, every character serves a purpose. The only problem is making sense of all that code later, remembering differences between fat and skinny arrow, and wrapping your head around all those invisible parentheses. CoffeeScript solved JavaScript’s verbosity problem, but introduced a new one. Its cryptic syntax is the reason many JavaScript developers recommend steering clear of it. Python has a good balance of the two, and as a result has gained quite a large following from developers who’re tired of dealing with the visual clutter of other languages. But, to be honest, it’s not as clear as modern JavaScript. Let me show you a few examples:

Python JavaScript Explanation
len(a) a.length Global scope, unnecessary abbreviation, should I say more?
def function Another vague abbreviation, but at least def makes it very clear what type of object we’re defining
lambda function At least they didn’t abbreviate it
*rest ...rest They went out of their way to rename all && and || and then introduced this?
**kw {foo:"bar"} Nothing says dictionary like 2 multiply signs next to each other
','.join(array) array.join(',') Are we joining elements via delimiter or delimiters via elements?
str String Another useless abbreviation that ignores the convention of capitalizing classes
list Array Linked list?
ZeroDivisionError Infinity Sure, it’s mathematically correct, but when I divide by zero, 9 times out of 10 infinity is what I want, I’ll end up assigning 999999 to the number in the catch block anyway

You may argue that Python’s operators are more clear than JavaScript’s ||, &&, but which one do most languages follow? Developers are so used to seeing those operators that it really doesn’t matter. RapydScript does as well as Python on signal, and sometimes a bit better (thanks to closures and ability to inject JS), I gave it an extra point for clarity because it can often leverage the cleaner JS approach, as I usually do in my code (although to be fair JavaScript still inches ahead a bit here).

Going Forward

So where does that leave us? As I said, I love RapydScript, but I don’t have the manpower to continue fixing the bugs or make RapydScript more like Python when I have no need for Python. My backends are all in node.js now. It also seems like Kovid has been more active in enabling Python users to feel at home with his fork, so I’ll let him continue that. At this point the philosophies have diverged somewhat. I disagree with adding extra syntax and special-casing variables for Python, he disagrees with breaking Python rules. At this point, I’m feeling like Python has become a bit of a shackle, there is no reason I should repeat the same mistakes as Guido in the name of backwards compatibility with code that wasn’t even written for the same platform.

My goal is to make RapydScript relevant in ES6 age, and here is how I plan to do that (at my own pace of course, given other projects, but you’re more than welcome to help me out):

  1. Fix low-hanging fruit bugs.
  2. Separate tokenizer from lexer (I’ll explain why in a bit).
  3. Rewrite output module to generate ES6 instead, let Babel handle special cases, not us.
  4. Modify output module to fill in templates rather than handcrafting output code.
  5. Drop pythonisms from core (no more int, str, bool) and embrace native JS types (Number, String, Boolean)
  6. Add optional static typing and ability to infer types, let users leverage this information to create/optimize better templates.
  7. Now that tokenizer is separate from lexer (step 2 accomplished this), add a macro-processing step in-between that can mutate the code as the AST is being formed, this will give RapydScript powers similar to sweet.js, but which can handle whitespace.

After all 7 of those steps are complete, users can enhance the language as they see fit, without having to modify the core. If they want a new operator, or pythonic variable names, it would be as simple as creating a macro. If they want different output/format, it would be as easy as writing an output template. Admittedly, this will probably take me a while. But if anyone wants to help, we can probably get this done much sooner.