Don't Mock Your Code - The Behavior is the Unit
I first started really utilizing unit testing and test-driven development while on the DataMarket team at Microsoft was slowing migrating an ASP.NET WebForms app to MVC. In a team that had no tests maintained by developers (shudder how did we develop software this long?) it was a big cultural change led by my great colleages Jon Sequiera and Dmitry Frenkel, who I am very greatful to have the experience of working with.
At the time we'd write tests for our controllers. We'd mock out all the dependencies it would need in a test and provide them. We were also using the repository pattern and testing the controller we'd mock that out. The repository pattern was really needed as we couldn't actually make a call to the database due to a lot of different reasons that basically stem from a monolithic ball of sphagetti code and abomination that was Entity Framework 1.0.
That was nice to get developers used to writing tests for their code, but there was a lot of problems with that. Views were not tested at all, due to Razor being untestable at the time (some 32bit vs 64bit issue on the build server or something, see previous comment on monolithic ball of sphagetti). We had bugs in our view not being rendered. Razor was also very logic heavy, and a few abominations were created. Writing new tests required about 3x the mocking code than code that actually did the assertions. Model binding had no tests - you'd know about it 3 days later (maybe) when a tester ran their test suite, or worse, your manager was trying something on the development environment. Forgot a dependency injection registration that you mocked out? App won't boot at all despite your tests passing. Moreover, when I want to refactor code to move more logic into the controller from the repository that had to much (that had no tests on it), I have to refactor a whole quite of tests. How can we rely on these tests when it still has these huge gaps and maintence problems?
Migration to a static site: Experience with Metalsmith vs Gulp
I started this site using poet on top of node.js and express, mainly wanting to expand my node skills and use my free Azure credits I got as a Microsoft employee. Azure Websites, the Heroku-like PaaS had just launched (now called App Services). It was a nice experience.
So how to build it?
Semantic Web Component Design: Radios, dropdowns, and tabs are the same thing
Early on in the rewrite of Appuri's portal that we started on after having a designer and development team from the MVP, and in our transition from Angular to Knockout, I was looking around at different widget / web component libraries and what I could leverage that would fit the designer's use cases and style, as well as a clean design.
One thing that always drove me nuts about our previous UI was a mismash of different components doing the same thing. We had ui-select2, ui-select from the Angular UI team, angular-multi-select for some multiselect cases, and a custom directive for radio-like toggle boxes.
Each control offered some little feature that another didn't (except
ui-select2 which was just in code we didn't get around to refactoring yet). Why is this? All of them are trying to do the same thing on a higher semantic level: The user needs to pick an one (or more) options from this fixed set of options. How that looks is a styling concern.
A fast and robust way to run Sauce Labs tests in Circle CI
We've been using Circle CI at Appuri for many months now and have been very happy with it. It's very simple and has a great breadth of features, namely the ability to SSH into a build to diagnose it when things get really tricky, as well as docker build services, slack integration, and lots of built-in services. After I hit a wall with PhantomJS, and needing to run our tests on multiple browsers anyways, it was time to integrate Sauce Labs to do our browser testing, which I've had great experience with on many open source projects. In those projects I used grunt-saucelabs as I am not a big fan of Karma as I like to control the HTML page the tests run on as well as the simplicity of mocha-phantomjs and just refreshing a flat file or using browser sync.
However we're using gulp and I didn't want to pull in grunt and have two build systems. Unfortunately there's no
gulp-saucelabs, which I contemplated doing (I may sometime), but I found I could simply use mocha-cloud. However, it didn't bring up the tunnel.
jspm and systemjs: High friction package managment
We have a decent-sized Angular codebase here at Appuri that I want to replace with real module system that will allow for a gradual move away from Angular. A friend recently pointed me to jspm and systemjs. The goal and direction is sound - load any module format (AMD, CommonJS, Harmony/ES6) and be registry-agnostic (npm or GitHub is out of the box with registry plugins, namely for bower). Unlike bower, since the loader and the package manager work together, it maintains the mapping of modules that normally you do manually in the
paths sections of your RequireJS main. It also has a builder, so you can work in a mode that loads files individually, or bundle them up together for production, with source maps and minifcation. It also supports ES6 compiling to ES5 (or "transpiling" if you don't fully understand what compiling is), so it is very future-standards focused. It's a superset of what I want from RequireJS, so I was excited and dove in.
Three days in, I'm ready to throw in the towel as I can't get SystemJS builder to work. There are bigger issues too.
Bower should be a first class registry
So jspm is registry agnostic, and it's registry providers built in are npm and GitHub. Why a GitHub provder instead of bower? It's so similar to bower, yet lacking too. They are both git based (well except these muts be in GitHub, damn your priviate or alternate git repo). It also ignores dependencies in the bower.json - which basically defeats the purpose of the package manager as you are now left to dealing with the dependecies yourself, manually editing your
config.js- the thing that it's supposed to manage for you.
I started out simply using
jspm install angular. Worked fine. Then trying to install angular plugins, they weren't in the registry. so I did them mangually via
jspm install github:somedude/angular-plugin. However they would often just have a bower.json, so the dependency on angular wasn't declared. It was worse as I did
angular-chart, which depends on
c3, which then depends on
Ugh. So I install jspm-bower-endpoint and run
jspm install bower:angular-chart. cool.... wait, it depends on
bower:angular, but I already have
github:angularinstalled from the jspm registry... Ugh.
jspm uninstallALL the jspm packages and just use the bower registry, otherwise unless the jspm registry is 100% complete, you'll just have a mess to manually maintain. The same is true if you mixed and matched
bower, unless the dependencies didn't cross. BTW, you still need to have a
jspmdown anyways.... so you have 2 package maangers. Actually, 2 package managers and a package manager registry wrapper.
Automatic module detection really does not work
Angular's module system fails at tracking dependencies properly
Angular, being a framework with a lot of solutions to a problem, forces users to use modules - a very good thing. However, rather than letting users choose their module and package system (RequireJS or Browserify or Webpack for loaders and AMD vs CommonJS vs Harmony for module formats), it brings in it's own module format, without a loader.
Defining a module looks like this
var module = angular.module('fruit-stand', ['raspberries', 'strawberries', 'apples']) // define what the module is or has by calling methods on it
Browserify string transforms: Use rfileify
I started to use browserify for the first time day. It is quite slick - Just
npm install something then
require it up, with a lot less wiring up needed than using AMD. However adding a forced build step is not fun and I think would really be a killer on a big project.
So I began to look for a transform so I could include templates as I needed to bundle everything into one script. How hard can it be to basically do
"module.exports = " + JSON.stringify(fileContents) ? Well after a couple hours of searching and trying 3 modules, the 4th one worked.
- stringify (no tests at all for this project)
- browserify-string (tests but no CI)
- string-to-jsify (test and CI but not for the console use case, the 95% one)
- rfileify (tests, CI, and mentioned from browserify-transform-tools)
Rfileify does other transforms too, but just not the SEO rank, so hopefully I can help that.
Why node.js is Awesome, Part 2: Package and Dependency Management
At the core of node is npm, the node package manager (which apparently npm doesn't stand for that). Having dependencies managed by a package system is huge. In Java and .NET, package managers have been added after the fact, if they are used at all. Often people end up checking in binaries that then makes the repository huge with information not pertinent to your product, which also results in binary merge conflicts. How do those get organized? A legacy codebase I work on has them scattered throughout 3 different locations, not to mention naming styles.
Npm standardizes this by having all the dependencies in a folder named node_modules which cannot be renamed and a package.json that must be comformant too, which then node understands when you
require something. No need to set classpaths or hint paths correctly.
This also heavily pushes people to have all their dependies locally. Its still possible to have a global module installed and require it without declaring it in your package.json (Maybe there is a way to prevent this via static analysis?) You can end up pulling assemblies from the GAC in .NET too, but still so many packages aren't installable via NuGet (I'm looking at you Azure SDK, which of course the Azure SDK for Node.js installs via npm) which means every developer and the build server(s) better have the same versions installed or someone will be wasting time getting stuff to build.
Dependencies are also a tree structure, not a graph. If two modules depend on the same module, they each get their own copy, so they can have their own versions. This is great given prototypical inheritance. If you want to be strict about the type of object you are working with,
instanceof will only be true if your version of the module created the object, not another version , due to the constructor functions not being a reference to the same object. If you want to be lenient, you can always duck type it.
Statically typed languages have a graph of dependencies, and if A and B depend on different versions of C, pain will ensue. I gave seen .NET developers waste hours tracking down mismatching versions of dependencies, even with the help of assembly binding redirects. I don't think Java has any answer for this.
- Model: Your data as an object
- Controller/ViewModel/Presenter/Whatever: the business logic of your app
View: The HTML templates for your app
But after using a few (Knockout, Angular, Ember, and Backbone) I've noticed there's a forth component. It's the widgety type of logic - logic that isn't about your business, but extents the browser functionality - that cool datepicker, color picker, an awesome visualization with D3, maybe a 3D flip switch intstead of a checkbox. That stuff that is a DMZ to the DOM - Knockout's
bindingHandlers, Angular's directives, and Handlebar helpers. They aren't about how you map your view model / controller to the DOM, but about extending the functionality of the DOM.
Why is this important? Keeping selector strings out of your controllers and view models is vital for the separation of concerns and tesability. Keeping this 4th aspect as the DMZ to the DOM helps to separate these concerns. Say you do have a complicated problem to solve in your app, and you come up with a cool way to visuallize it. You could put it all in one module... but now what happens when you release it and users don't like that widget and they want something more traditional? It will be harder to pull it out. Or the opposite where that widget is really awesome and you want to use it elsewhere... now lots of work to pull the business logic out. Let those two evolve independently and they grow and improve much more rapidly.
Why node.js is Awesome, Part 1: The Single Responsibility Principle
My favorite thing about node.js and the npm community is something that can't easily be put on a spec sheet. It's adherance to the single responsiblity principle.
Take a look at some of the most depnended on npm modules - they're often very small, like mkdirp, which is simply
mkdir -p in node. glob matches files like a shell does. node-uuid just generates UUIDs.
Is there a bug in the module? The module is small enough and tested well that you can dive in and help out, and send the bug fix as a pull request on GitHub. Don't like the design of the module your using, or development died out? Swap it out with a different module, and wire it up to your loosely coupled system.
This is definately a change in culture, as though I didn't use Rails much, but that community likes it's big frameworks that do a lot of magic. Which is nice until you disagree with an aspect of that magic, or need to tweak it or swap it out with something else for this one scenario.
A great coworker of mine once told me "The key to complexity is composition", which I have experienced to be true.
After owning nathanblack.org for several years and doing web development, it's long overdue I have a blog with a few thoughts to share about web development.
This blog is powered by poet on top of node.js and express.js, with stylus for the CSS preprocessor, but most importantly no jQuery and no bootstrap.
Update: See the entry about the current setup