Chris Callahan

React.Children and the React Top-Level API

Fri, 16 Sep 2016 18:06:19 +0000

One of the features that drew me in to React was the explicit intention by its authors to create an API with a minimal surface area. This is a great goal, because it allows you to keep the full React API in your head when developing your application with React. Writing React code is often focused on creating components and using the component API and lifecycle methods, but there are a few other useful top-level utilities provided in the React API. Below I’ll cover the React.Children methods for handling the this.props.children opaque data structure and how I’ve used them in my React apps.

`React.Children.map`

The release notes for React.cloneElement discuss a common pattern that consists of using React.Children.map to map over the children prop passed to a component and returning a cloned version of each child with additional props passed in. This allows you to pass props down from a parent component to a child component without the parent needing to explicitly render that child. This can be a big help in building reusable React components.

import React from 'react';

const Salmonize = ({ children }) => (
  <div>
    {React.Children.map(children, child => (
      React.cloneElement(child, {
        style: {
          backgroundColor: 'salmon',
          color: 'seagreen',
        }
      })
    ))}
  </div>
);

const SalmonBlog = ({ title, posts }) => (
  <div>
    <Salmonize>
      <NavBar title={title} />
    </Salmonize>
    {posts.map(post => (
      <Post key={post.id}>
        <Salmonize>
          <PostHeader title={post.title} />
        </Salmonize>
        <PostBody text={post.text} />
      </Post>
    ))}
  </div>
);

In this example, the <NavBar> and <PostHeader> elements don’t need to know about their styles, and indeed these components could be used elsewhere without this salmon-inspired color palette. (CSS is also a solution here 😄.) Similarly, the <Salmonize> component does not need to be aware of the fact that it is rendering the <Navbar> or <PostHeader> components, but it is still able to pass its props along to those child components.

It’s definitely a bit tricky to write truly reusable components in React. If you’re having trouble with this, then this “map and clone” trick can probably help you on the way to achieving this elusive goal.

`React.Children.forEach`

Similar to React.Children.map, but has no return value. I haven’t used this method as I haven’t had the need to loop over a component’s children and do something other than augment props or render UI. There is a pretty elegant use case for this in Pete Hunt’s rwb repo here.

`React.Children.count`

This method returns the number of child components in the children argument passed to it. Again, I haven’t come across the use case for this one personally. Most of the use cases I came across in a cursory Github search were simply confirming whether or not any children had been passed to a component, as in the above example from Pete Hunt.

`React.Children.only`

This can come in handy when you want to ensure that a component only has one child, and throw an error if this condition is not met. There’s a good example of this in a previous version of Cheng Lou’s react-radio-group component here:

export default React.createClass({
  // ...

  render: function() {
    const {name, selectedValue, onChange, children} = this.props;
    const renderedChildren = children(radio(name, selectedValue, onChange));
    return renderedChildren && React.Children.only(renderedChildren);
  }
});

The example app in the repo complies with this contract.

`React.Children.toArray`

This method converts the children prop of a component into a plain JavaScript array, which can give you a bit more flexibility than React.Children.map provides. React.Children.toArray came in handy for me recently when I wanted to render a list of items with a divider element interspersed between them. This led me to create an <IntersperseDividers> component to accomplish just that.

import React from 'react';

const IntersperseDividers = ({ children }) => (
  <div>
    {React.Children.toArray(children).reduce((elements, child, i, array) => {
      elements.push(child);
      if (i < array.length - 1) {
        elements.push(<hr key={`${i}--divider`} />);
      }
      return elements;
    }, [])}
  </div>
);

const List = ({ data }) => (
  <IntersperseDividers>
    {data.map((item, i) => (
      <div key={i}>
        {item.value}
      </div>
    ))}
  </IntersperseDividers>
);

I got caught on this component for a moment because of an error thrown when I tried to directly return an array of elements. I fixed it by wrapping the array returned by React.Children.toArray() in a <div>, just like in the React.Children.map example above. You wouldn’t need this wrapping <div> if the content was inlined in the <List> component, so it’s a tradeoff for better component reusability at the expense of increased divitis.

Conclusion

If you haven’t read through it before, I’d recommend taking a minute to skim through the top-level API section of the React docs. Though the API surface area is small, there are some hidden gems within.

Debugging CouchDB Design Documents

Sat, 04 Jun 2016 19:40:09 +0000

When making changes to a CouchDB design document (or ddoc), I often use curl to make queries and see if the data returned is what I expected. Sometimes, however, this does not provide enough information to help me get to the bottom of why the ddoc does not behave as anticipated. Here is my process for debugging JavaScript code in CouchDB design documents. (Note: This blog post assumes CouchDB is installed and running on your machine.)

TL;DR: I show you how to use console.log debugging inside of CouchDB design documents.

How to Configure CouchDB

The first step is to locate CouchDB’s configuration settings on your machine. Type couchdb -c into the command line:

$ couchdb -c
/usr/local/etc/couchdb/default.ini
/usr/local/etc/couchdb/local.ini
/usr/local/etc/couchdb/local.d/couchdb-lucene.ini

This output is a list files containint of CouchDB’s default configurations and any custom overrides you define. Open the local.ini file in your preferred text editor and navigate down to the [log] section. You may see the following text:

[log]
;level = debug

The leading semicolon indicates this line is commented out. I am not totally sure why CouchDB uses .ini files for its config files, as the database is primarily written in Erlang and contains a JavaScript runtime (SpiderMonkey).

Uncomment this line (delete the semicolon), restart CouchDB, and you will see quite a bit of output in the console, starting with something like this:

$ couchdb
Apache CouchDB 1.6.1 (LogLevel=debug) is starting.
Configuration Settings ["/usr/local/etc/couchdb/default.ini",
                        "/usr/local/etc/couchdb/local.ini",
                        "/usr/local/etc/couchdb/local.d/couchdb-lucene.ini"]:
# About 100 more lines of lots of nice config info follows

This being CouchDB, we can of course set the log level using Couch’s HTTP API:

$ curl -X PUT 127.0.0.1:5984/_config/log/level -d '"info"'
"info"

(Note that you will have to pass in admin credentials here if the CouchDB you’re running is not in admin party mode.)

The best part about this is that you don’t have to restart the database, logging will begin immediately. Thanks, Erlang!

The CouchDB configuration settings can also be changed in the admin UI of Futon. Visit http://localhost:5984/_utils, log in if necessary, then click on “Configuration” on the righthand panel. If this looks familiar, it is because it is a web interface for the local config files listed when you ran couchdb -c above. You can edit the log level here as well – navigate down to the “log” section, double click on the text next to “level” (should be “info”), type in “debug”, and click the green check mark. You’ll notice the following output gets logged by CouchDB:

=SUPERVISOR REPORT==== 4-Jun-2016::15:23:18 ===
     Supervisor: {local,couch_primary_services}
     Context:    child_terminated
     Reason:     normal
     Offender:   [{pid,<0.258.0>},
                  {id,couch_log},
                  {mfargs,{couch_log,start_link,[]}},
                  {restart_type,permanent},
                  {shutdown,brutal_kill},
                  {child_type,worker}]

[info] [<0.157.0>] 127.0.0.1 - - PUT /_config/log/level 200

The last line shows that under the hood the Futon UI is simply making the same PUT request over HTTP you made above using curl to change the log level. Above that there is a “Supervisor Report” from the CouchDB internals stating that the couch_log worker process was… abruptly?… restarted by the couch_primary_services supervisor process.

The CouchDB docs has a good roundup of all of the different log levels Couch supports. I typically stick with “info” in development to have a window into all of the incoming requests to CouchDB being logged in real time, but I’ll sometimes switch to “debug” mode to view in more detail all of the headers / cookies that are part of the incoming HTTP requests. (Note that neither of these settings are recommended for logging in production.)

Now that you’ve set up CouchDB to be a bit more verbose, run:

curl 127.0.0.1:5984

Back in CouchDB, you’ll see the following output:

[debug] [<0.225.0>] 'GET' / {1,1} from "127.0.0.1"
Headers: [{'Accept',"*/*"},
          {'Host',"127.0.0.1:5984"},
          {'User-Agent',"curl/7.43.0"}]
[debug] [<0.225.0>] OAuth Params: []
[info] [<0.225.0>] 127.0.0.1 - - GET / 200

Simpy perusing the logs of incoming requests and outgoing responses can be very informative and helpful if something is going wrong. But you can also take the debugging one step further with log.

“`console.log` Debugging” Inside CouchDB

CouchDB exposes a global log function inside of design documents that allows you to write custom messages to CouchDB’s logs. This is similar to using console.log() in the dev tools console in the browser or in node.

To see how this works in action, first create a new database and insert a design document:

$ curl -X PUT 127.0.0.1:5984/test
{
  "ok": true
}
$ curl -X PUT 127.0.0.1:5984/test/_design/test \
> -d '{ "_id": "_design/test", "views": { "greetingDocs": { "map": "function(doc) { if (doc.hello) { log(\"This doc says hello, \" + doc.hello); emit(doc._id, null); } }", "reduce": "_count" } } }'
{
  "ok": true,
  "id": "_design/test",
  "rev": "1-ea73ae4cba8ca3ca7618c23185ef92d2"
}

Here’s roughly how that ddoc looks in JavaScript (sans stringified functions):

{
  _id: "_design/test",
  views: {
    greetingDocs: {
      map: function(doc) {
        if (doc.hello) {
          log("This doc says hello, " + doc.hello);
          emit(doc._id, null);
        }
      },
      reduce: "_count"
    }
  }
}

Now that you’ve got a barebones design doc, you can see the logging at work by adding a new document to the test database:

$ curl -X PUT 127.0.0.1:5984/test/one \
> -d '{ "_id": "one", "hello": "I am log" }'
{
  "ok": true,
  "id": "one",
  "rev": "1-6feff67ca27f6cc570f9ae43385730f8"
}
$ curl 127.0.0.1:5984/test/_design/test/_view/greetingDocs
{
  "rows": [
    { "key": null, "value" : 1 }
  ]
}

In the couch logs you will see something like this:

[debug] [<0.99.0>] 'GET' /test/_design/test/_view/greetingDocs {1,1} from "127.0.0.1"
Headers: [{'Accept',"*/*"},
          {'Host',"127.0.0.1:5984"},
          {'User-Agent',"curl/7.43.0"}]
[debug] [<0.99.0>] OAuth Params: []
[info] [<0.991.0>] Opening index for db: test idx: _design/test sig: "eff5145ec110e4f61dbe8a2b446588e8"
[info] [<0.996.0>] Starting index update for db: test idx: _design/test
[debug] [<0.715.0>] OS Process #Port<0.3200> Input  :: ["reset",{"reduce_limit":false,"timeout":5000}]
[debug] [<0.715.0>] OS Process #Port<0.3200> Output :: true
[debug] [<0.715.0>] OS Process #Port<0.3200> Input  :: ["add_fun","function(doc) { if (doc.hello) { log(\"This doc says hello, \" + doc.hello); emit(doc._id, null); } }"]
[debug] [<0.715.0>] OS Process #Port<0.3200> Output :: true
[debug] [<0.84.0>] New task status for <0.1001.0>: [{changes_done,3},
                                                    {database,<<"test">>},
                                                    {design_document,
                                                     <<"_design/test">>},
                                                    {progress,150},
                                                    {started_on,1465068874},
                                                    {total_changes,2},
                                                    {type,indexer},
                                                    {updated_on,1465068874}]
[debug] [<0.715.0>] OS Process #Port<0.3200> Input  :: ["map_doc",{"_id":"one","_rev":"1-6feff67ca27f6cc570f9ae43385730f8","hello":"I am log"}]
[debug] [<0.715.0>] OS Process #Port<0.3200> Output :: ["log","This doc says hello, I am log"]
[info] [<0.715.0>] OS Process #Port<0.3200> Log :: This doc says hello, I am log
[debug] [<0.715.0>] OS Process #Port<0.3200> Output :: [[["one",null]]]
[debug] [<0.991.0>] Updated index for db: test idx: _design/test seq: 2
[info] [<0.996.0>] Index update finished for db: test idx: _design/test
[debug] [<0.991.0>] Updated index for db: test idx: _design/test seq: 2
[info] [<0.99.0>] 127.0.0.1 - - GET /test/_design/test/_view/greetingDocs 200

It looks a bit cryptic, but towards the bottom you might notice four lines in particular:

[debug] [<0.715.0>] OS Process #Port<0.3200> Input  :: ["map_doc",{"_id":"one","_rev":"1-6feff67ca27f6cc570f9ae43385730f8","hello":"I am log"}]
[debug] [<0.715.0>] OS Process #Port<0.3200> Output :: ["log","This doc says hello, I am log"]
[info] [<0.715.0>] OS Process #Port<0.3200> Log :: This doc says hello, I am log
[debug] [<0.715.0>] OS Process #Port<0.3200> Output :: [[["one",null]]]

On the third line (with Log ::) we can see our message from log! It’s also interesting to see the Input and Output CouchDB is writing here. First, in the Input line, the doc with _id of "one" that you just PUT to the database is being run through something called map_doc. This produces two Outputs: first the log, and then the doc’s _id and null, which were emitted using emit(doc._id, null) in the map function of the greetingDocs view.

Running the same query on the view again will not write the same message to the log. My speculation is that CouchDB caches the results of the map and reduce functions in its views, so the map function won’t be run again (and therefore our message won’t be logged again) until either the ddoc changes or a new document is added to / updated in the database.

Let’s update the doc to test this assumption:

$ curl -X PUT 127.0.0.1:5984/test/one \
> -d '{ "_id": "one", "_rev": "1-6feff67ca27f6cc570f9ae43385730f8", "hello": "this is log()" }'
{
  "ok": true,
  "id": "one",
  "rev": "2-83ba9fe4df631f7e1af0b079baf1948b"
}

Indeed, when we query the view again, the CouchDB logs contain what we’re looking for:

$ curl 127.0.0.1:5984/test/_design/test/_view/greetingDocs
{
  "rows": [
    { "key": null, "value" : 1 }
  ]
}

[debug] [<0.715.0>] OS Process #Port<0.3200> Input  :: ["map_doc",{"_id":"one","_rev":"2-83ba9fe4df631f7e1af0b079baf1948b","hello":"this is log()"}]
[debug] [<0.715.0>] OS Process #Port<0.3200> Output :: ["log","This doc says hello, this is log()"]
[info] [<0.715.0>] OS Process #Port<0.3200> Log :: This doc says hello, this is log()
[debug] [<0.715.0>] OS Process #Port<0.3200> Output :: [[["one",null]]]

Wrapping up

log is a convenient way to quickly do console.log debugging inside of your CouchDB design documents. While a full-fledged debugger REPL like the one shown in this video seems quite amazing, you can still get quite far with good old log!

Resources for Learning React & Flux

Fri, 16 Jan 2015 16:34:16 +0000

I’ve recently had a few people ask me about resources for learning React.js and Flux, so I figured I’d put all of the resources I’ve used in one spot. Please leave a comment if you find any of these useful/useless or have any good resources I’ve left out!

Why React?

The unexamined 3rd party library is not worth using

React Basics

React 101

Flux

Facebook’s architecture pattern for organizing large React apps

Shameless plug: My Intro to Flux blog post – some helpful links on the bottom too
Flux for Stupid People
Facebook’s Flux Overview
Facebook’s Flux chat sample app – great code to dive into for a standard implementation of Facebook-flavored Flux

React Router

Inspired by Ember Router, a great addition to React apps

Server-Side Rendering

Learn the way of React.renderToString()

Tutorials

Learn by doing

Make Hacker News with React
Scotch.io React & Flux tutorials:
John Biesnecker’s React Tutorials:
1. Building a card game (war) with React
2. Using React.js to draw dynamic SVGs

Cool React Projects

React rocks

react.rocks

Official Documentation

Still the best and most comprehensive resource out there

Read the Source Code

Your great changes to React & Flux are just a pull request away…

Array Iteration Methods in JavaScript

Sat, 03 Jan 2015 20:55:19 +0000

When I first started learning JavaScript, I went through a number of tutorials that showed how to iterate over collections in an imperative style using a for loop:

var angLeeMovies = ['Pushing Hands', 'The Wedding Banquet', 'Eat Drink Man Woman'], i;

for (i = 0; i < angLeeMovies.length; i += 1) {
  console.log(angLeeMovies[i] + ' was directed by Ang Lee.');
}

// "Pushing Hands was directed by Ang Lee."
// "The Wedding Banquet was directed by Ang Lee."
// "Eat Drink Man Woman was directed by Ang Lee."

Before ES5, this was the best available way to iterate over an array, but now there are a number of more expressive, idiomatic, and functional ways of iterating over JavaScript arrays.

The Big Three: `map()`, `filter()` & `reduce()`

`Array.prototype.map()`

map() is the array method I find myself using all the time when writing JavaScript code. The conceit is simple: map() takes a callback function as a parameter, calls this function on each item in the array, and returns a new array of the same length as the initial array that contains the resulting values.

var angLeeMovies = [
    {title: 'Life of Pi', year: 2012},
    {title: 'Taking Woodstock', year: 2009},
    {title: 'Lust, Caution', year: 2007},
    {title: 'Brokeback Mountain', year: 2005},
    {title: 'Hulk', year: 2003}
  ],
  releaseYears;

releaseYears = angLeeMovies.map(function(movie) {
  return movie.year;
});

console.log(releaseYears); // [2012, 2009, 2007, 2005, 2003]

This can be simplified even further by extracting the function and passing it as an argument to map().

var angLeeMovies = [ ... ], releaseYears;

function extractYear(movie) {
  return movie.year;
}

releaseYears = angLeeMovies.map(extractYear);

console.log(releaseYears); // [2012, 2009, 2007, 2005, 2003]

Here movie is being implicitly passed as an argument to the extractYear() function, but the behavior is the same as above.

MDN Documentation - Array.prototype.map()

`Array.prototype.filter()`

filter() is intuitively named: it calls a callback function on each item in an array, filters out items that return a falsy value, and returns a new array with the remaining items (those that the callback returns a truthy value for).

var truthyAndFalsy = [-0, 1, 0, 2, undefined, 3, null, 4, "", 5, NaN, 6, false, 7], truthy;

truthy = truthyAndFalsy.filter(function(item) {
  return item;
});

console.log(truthy); // [1, 2, 3, 4, 5, 6, 7]

Using the opposite test case, we can see the seven falsy values in JavaScript:

var truthyAndFalsy = [ ... ], falsy;

falsy = truthyAndFalsy.filter(function(item) {
  return !item;
});

console.log(falsy); // [-0, 0, undefined, null, "", NaN, false]

While 0 being falsy is a “classic JavaScript gotcha”, the point remains that filter() is a powerful tool for collecting a subset of your data that matches certain criteria.

MDN Documentation - Array.prototype.filter()

`Array.prototype.reduce()`

reduce() has been the hardest of all FP methods for me to fully embrace, but it can be extremely useful in certain cases. The core idea of reduce() is that it iterates over an array and returns exactly one value. The return value is commonly a sum or count of some property of the items in the array, but reduce() is quite flexible in what it can produce.

var angLeeMovies = [
    {title: 'Brokeback Mountain', bestDirectorOscar: 'won'},
    {title: 'Crouching Tiger, Hidden Dragon', bestDirectorOscar: 'nominated'},
    {title: 'Hulk', bestDirectorOscar: false},
    {title: 'Life of Pi', bestDirectorOscar: 'won'},
    {title: 'The Ice Storm', bestDirectorOscar: false}
  ],
  bestDirectorWins;

bestDirectorWins = angLeeMovies.reduce(function(previousValue, currentValue) {
  return currentValue.bestDirectorOscar === 'won' ? previousValue + 1 : previousValue;
}, 0);

console.log(bestDirectorWins); // 2

Here the previousValue argument is the accumulator that we increment if Ang Lee won an Oscar for best director for the movie we’re iterating over, and the currentValue argument is item being iterated over. As there is no previousValue when iterating over the first item, we can initialize it to 0 as an optional second parameter to the reduce() function.

Though it can seem somewhat esoteric at first, reduce() has grown on me significantly in the past few months – partly because of using MapReduce views in CouchDB views, and partly because of learning more about the benefits of functional programming.

MDN Documentation - Array.prototype.reduce()

Friends of `reduce()`

`Array.prototype.reduceRight()`

Having grasped reduce(), it is not a far stretch to get reduceRight(). Whereas reduce() iterates over an array from left to right, reduceRight() iterates from right to left. This does not make a difference for the standard sum/count use cases, but it can come in handy for other purposes. For example, if we have an array of Ang Lee movies sorted in reverse chronological order, we could use reduceRight() to sort these movies in chronological order by decade.

var angLeeMovies = [
    {title: 'Life of Pi', year: 2012},
    {title: 'Brokeback Mountain', year: 2005},
    {title: 'Crouching Tiger, Hidden Dragon', year: 2000},
    {title: 'The Wedding Banquet', year: 1993}
  ],
  moviesByDecade;

function addMovieToDecade(filmography, decade, title) {
  filmography[decade] ? filmography[decade].push(title) : filmography[decade] = [title];
  return filmography;
}

moviesByDecade = angLeeMovies.reduceRight(function(previousValue, currentValue) {
  if (currentValue.year >= 2010) {
    return addMovieToDecade(previousValue, "2010s", currentValue.title);
  } else if (currentValue.year >= 2000) {
    return addMovieToDecade(previousValue, "2000s", currentValue.title);
  } else {
    return addMovieToDecade(previousValue, "1990s", currentValue.title);
  }
}, {});

console.log(moviesByDecade);
// {
//   "1990s": ["The Wedding Banquet"],
//   "2000s": ["Crouching Tiger, Hidden Dragon", "Brokeback Mountain"],
//   "2010s": ["Life of Pi"]
// }

MDN Documentation - Array.prototype.reduceRight()

`Array.prototype.every()`

every() checks to see if all items in an array return a truthy value for the callback function. If they all do, every() returns true; otherwise, it returns false. Given what we know about reduce(), it seems like every() is achieving a similar purpose: boiling an array down to a single value. It is therefore pretty straightforward to implement our own version of every() based on reduce(). (Note, this is a simplified version, as every() can take an optional thisArg argument).

Array.prototype.myEvery = function(callback) {
  return this.reduce(function(previousValue, currentValue) {
    return previousValue ? callback(currentValue) : false;
  }, true); // every() returns true for empty arrays
};

[1, 2, 3].myEvery(function(num) { return num < 10; }); // true
[1, 2, 3].myEvery(function(num) { return num < 3; });  // false
[1, 2, 3].myEvery(function(num) { return num < 0; });  // false

Unlike my crude implementation, every() is optimized to return immediately if a falsy value is returned from the callback. Personally, I haven’t used every() much – I prefer to use map(), filter(), and reduce().

MDN Documentation - Array.prototype.every()

`Array.prototype.some()`

Similar to every(), some() iterates over an array and returns true if any items in the array return a truthy value, and false if none do. This can also be (again, crudely) expressed in terms of reduce():

Array.prototype.mySome = function(callback) {
  return this.reduce(function(previousValue, currentValue) {
    return previousValue || callback(currentValue);
  }, false); // some() returns false for empty arrays
};

[1, 2, 3].mySome(function(num) { return num < 10; }); // true
[1, 2, 3].mySome(function(num) { return num < 3; });  // true
[1, 2, 3].mySome(function(num) { return num < 0; });  // false

MDN Documentation - Array.prototype.some()

A More Expressive `for` Idiom?

`Array.prototype.forEach()`

To me, forEach() appears to be a direct successor to the for (i = 0; i < x.length; i += 1) {} example above. It is a basic iterator that gives you access to one item in an array at a time, and returns undefined.

var angLeeMovies = [
    {title: 'Brokeback Mountain', bestDirectorOscar: 'won'},
    {title: 'Crouching Tiger, Hidden Dragon', bestDirectorOscar: 'nominated'},
    {title: 'Hulk', bestDirectorOscar: false},
    {title: 'Life of Pi', bestDirectorOscar: 'won'},
    {title: 'The Ice Storm', bestDirectorOscar: false}
  ];

angLeeMovies.forEach(function(movie) {
  if (movie.bestDirectorOscar === 'won') {
    console.log('Ang Lee won an Oscar for best director for ' + movie.title);
  } else if (movie.bestDirectorOscar === 'nominated') {
    console.log('Ang Lee was nominated for an Oscar for best director for ' + movie.title);
  }
});

// "Ang Lee won an Oscar for best director for Brokeback Mountain"
// "Ang Lee was nominated for an Oscar for best director for Crouching Tiger, Hidden Dragon"
// "Ang Lee won an Oscar for best director for Life of Pi"
// undefined

Unlike the previous examples, forEach() is used to produce “side effects”. Practically speaking, because the return value of forEach() is always undefined, it can only be used to change the state of or interact with other parts of the program. This may seem like a fine distinction to some, but the use of side effects is a hotly contested debate across many programming language communities.

It is worth pointing out that forEach() is actually significantly slower than the for (i = 0; i < x.length; i += 1) {} idiom. But for most purposes, forEach() is a much more expressive way to iterate over collections.

MDN Documentation - Array.prototype.forEach()

Conclusion

Iterating over arrays is a common task when coding in almost any language, and JavaScript is no exception. The above Array.prototype iterators are all defined in the ES5 spec and are widely supported on most modern web browsers (including IE 9 and above). They are relatively straightforward to use and make coding in JavaScript more functional and expressive.

Resources

88 mph With Flux and React

Mon, 24 Nov 2014 03:41:11 +0000

React.js has gained a lot of attention since it was open sourced by Facebook last year, and with its recent version 0.12 release, it appears that a version 1.0 may soon be on the horizon. React is a lightweight front end toolkit for handling the view layer of web applications that emphasizes a unidirectional data flow to simplify logic about state and data binding. Under the hood, React utilizes a virtual DOM, diffing this with the plain old DOM (PODOM) and rerendering the page with the minimal effort necessary to reflect the updated state of the application. This is an interesting abstraction that makes it easier to reason about state in an application and relieves the need for developers to directly manipulate the DOM.

This year, Facebook introduced Flux, the application architecture they use for React apps. Flux is more of a pattern for separating concerns and organizing files in large React applications than it is a full-blown front end framework like Angular or Ember. Flux provides a roadmap for ensuring the unidirectional flow of data in apps and allows for more modular processing of state changes, event handling, and interfacing with external services.

I think React in particular is relatively easy and intuitive to understand. It provides a sparse API that makes it quick to set up a few components, attach some event listeners to the DOM, and get an app up and running. JSX, a language that serves up a sugary mix of HTML and interpolated JavaScript (it compiles to JavaScript), is a bit odd at first. It’s quite easy to get used to though, and remarkably it totally relieves you from the need to do string concatenation, which is a big plus.

Flux, on the other hand, takes a bit more time to fully grok. The conceit is simple: all data flows in one direction. But as I had previously primarily developed apps using the MVC architecture, Flux was a bit of a tough nut to crack.

In this blog post, I hope to make it clear how to get started using Flux in a React application.

Conceptual Model

Before I get into building the example app, it’s important to take a moment to think about the big picture of Flux. Luckily, there is one phenomenal chart that shows how all of the pieces work together:

The key parts of a Flux application are:

Actions
Dispatcher
Stores
Controller-View
Views

Lesser roles are played by:

Constants
Utils
Router

Keep in mind, this is all on the front end. React and Flux were explicitly intended to be back end agnostic. Plug in your favorite database and back end framework at will, should the need arise.

Architecture

Let’s say we want to build a small app that keeps track of the speed of Doc Brown’s DeLorean from Back to the Future, and lets users know if the flux capacitor has been activated yet – the deciding factor of whether or not time travel is possible at their current speed. Feel free to follow along with the project repo on github.

Each of the key parts mentioned above gets their own folder in the file directory structure. A barebones Flux and React app (i.e. exclusively HTML, CSS, and JS files) should look something like this:

.
├── css
|   └── styles.css
├── js
|   ├── actions
|   |   └── DeLoreanActionCreators.js
|   ├── components
|   |   ├── Accelerator.js
|   |   ├── DeLorean.js
|   |   ├── ImageSection.js
|   |   └── Speedometer.js
|   ├── constants
|   |   └── AppConstants.js
|   ├── dispatcher
|   |   └── AppDispatcher.js
|   ├── stores
|   |   └── DeLoreanStore.js
|   ├── app.js
|   └── bundle.js (this will be compiled automatically)
├── .gitignore
├── index.html
└── package.json

I’ll use NPM for a package manager here. It is also easy to drop in a link to Facebook’s CDN or download React. Be aware that this does not include Flux, and with larger applications it makes sense to use a package manager where we can pull in several dependencies.

Here’s the package.json file:

{
  "name": "bttf",
  "version": "0.0.1",
  "description": "Back to the Future-themed Flux + React app",
  "repository": "https://github.com/callahanchris/bttf",
  "main": "js/app.js",
  "dependencies": {
    "flux": "^2.0.1",
    "object-assign": "^1.0.0",
    "react": "^0.12.1"
  },
  "devDependencies": {
    "browserify": "^6.3.2",
    "reactify": "^0.17.0",
    "watchify": "^2.1.1"
  },
  "scripts": {
    "watch": "watchify -o js/bundle.js -v -d .",
    "build": "browserify js/app.js -o js/bundle.js"
  },
  "author": "Chris Callahan",
  "browserify": {
    "transform": [
      "reactify"
    ]
  }
}

the index.html file:

<!DOCTYPE html>
<html>
  <head>
    <meta charset="utf-8">
    <title>88 mph With Flux and React</title>
    <link rel="stylesheet" href="css/styles.css">
  </head>
  <body>
    <section id="bttf"></section>
    <script src="js/bundle.js"></script>
  </body>
</html>

and the js/app.js file to bootstrap this app:

var React = require('react');
var DeLorean = require('./components/Delorean');

var App = React.render(
  <DeLorean />,
  document.getElementById('bttf')
);

For now we’ll use just one CSS file, css/styles.css. However, if the app gets big enough we might want to make the CSS more modular. One way we could do this is by using individual CSS files for each React component, (e.g. css/Accelerator.css, css/DeLorean.css, etc.).

Writing the App… The Flux Way

Although the actions are at the top of the foodchain, it helps to conceptualize the flow of a Flux app starting from the View layer.

At the top level of the View layer we have the “Controller-View”. For our purposes, this will be the DeLorean component that will wrap the entire application. The Controller-View is responsible for registering event listeners with the Stores, retrieving state from the Stores when the Stores emit an event, rerending the application with this new state using this.setState(), and passing the state down to nested components through props.

// js/components/DeLorean.js

var React = require('react');
var Accelerator = require('./Accelerator');
var ImageSection = require('./ImageSection');
var Speedometer = require('./Speedometer');
var DeLoreanStore = require('../stores/DeLoreanStore');

var DeLorean = React.createClass({
  getInitialState: function() {
    return {speed: 0};
  },

  componentDidMount: function() {
    DeLoreanStore.addChangeListener(this._onChange);
  },

  componentWillUnmount: function() {
    DeLoreanStore.removeChangeListener(this._onChange);
  },

  fluxCapacitorActivated: function() {
    return this.state.speed >= 88;
  },

  render: function() {
    return (
      <div className="delorean">
        <ImageSection />
        {this.fluxCapacitorActivated() ? '' : <Accelerator />}
        <Speedometer speed={this.state.speed} />
      </div>
    );
  },

  _onChange: function() {
    this.setState({speed: DeLoreanStore.getSpeed()})
  }
});

module.exports = DeLorean;

Say that the Accelerator component contains a <button> element. Unable to resist the temptation, a user clicks the button. This triggers the onClick event handler we have embedded in this button, which calls the handleClick method defined on the Accelerator component.

// js/components/Accelerator.js

var React = require('react');
var DeLoreanActionCreators = require('../actions/DeLoreanActionCreators');

var Accelerator = React.createClass({
  handleClick: function() {
    DeLoreanActionCreators.accelerate(1); // hard-coded to show how data flows through Flux
  },

  render: function() {
    return (
      <div className='accelerator'>
        <button onClick={this.handleClick}>Activate the Flux Capacitor</button>
      </div>
    );
  }
});

module.exports = Accelerator;

This method sends a message to an Action Creator. Action Creators supply an API of accepted methods that can be invoked by components, and are also the place where information can break out of the Flux loop and communicate via XHR with the back end, external APIs, a database, etc. In the case of this app, it would be a bit contrived to ping an API or a back end, so I will keep the DeLoreanActionCreators file super simple and handle the logic in the DeLoreanStore.

// js/actions/DeLoreanActionCreators.js

var AppDispatcher = require('../dispatcher/AppDispatcher');
var ActionTypes = require('../constants/AppConstants').ActionTypes;

module.exports = {
  accelerate: function(mph) {
    AppDispatcher.handleViewAction({
      type: ActionTypes.ACCELERATE,
      mph: mph
    });
  }
};

Action Creators can pass on any updated state they receive from the components to the Dispatcher along with a set action type. Typicially the name of the invoked method on the Action Creator mirrors the name of the action type. Action types are listed in the constants folder.

// js/constants/AppConstants.js

module.exports = {
  ActionTypes: {
    ACCELERATE: 'ACCELERATE'
  },

  PayloadSources: {
    SERVER_ACTION: 'SERVER_ACTION',
    VIEW_ACTION: 'VIEW_ACTION'
  }
};

This set of SCREAMING_CAMEL_CASE constants serves as the one definitive source of what actions are legal in the Flux application. Don’t overthink this – the sole purpose of the constants folder is for looking up constants. If the app gets large enough, you can get more modular in how you classify the constants, but at the end of the day all you want to know is what exactly are the officially sanctioned actions in this app.

Next, the AppDispatcher receives the handleViewAction() message along with the data above, which is referred to as the method argument action below. The AppDispatcher is a singleton in the Flux app and has two main functions: forwarding the data it receives from the actions to all stores, and keeping track of the callbacks registered by stores.

The AppDispatcher typically responds to messages sent from the Action Creators by calling the dispatch() method on itself.

// js/dispatcher/AppDispatcher.js

var Dispatcher = require('flux').Dispatcher;
var PayloadSources = require('../constants/AppConstants').PayloadSources;
var assign = require('object-assign');

var AppDispatcher = assign(new Dispatcher(), {
  handleViewAction: function(action) {
    var payload = {
      source: PayloadSources.VIEW_ACTION,
      action: action
    };
    this.dispatch(payload);
  },

  handleServerAction: function(action) {
    var payload = {
      source: PayloadSources.SERVER_ACTION,
      action: action
    };
    this.dispatch(payload);
  }
});

module.exports = AppDispatcher;

Dead simple. I think the choice of wording here was good, as it’s just like an emergency dispatcher that broadcasts messages to EMTs in the field and keeps track of who’s on duty.

Creating Dispatchers used to require a lot of boilerplate code, but a canonical Dispatcher has recently been packaged up as part of the Flux NPM module. Check out the Dispatcher’s seriously simple builtin API in the offical documentation to see what goes on under the hood here.

In a front end app, Stores are responsible for the logic that we typically associate with a model on the back end. Stores inherit from the prototype of node’s EventEmitter, and are responsible for consuming the data passed down from the dispatcher, changing its state internally, and emitting an event which forces the whole app to update its state.

As I mentioned above, Stores also are required to register their callbacks with the AppDispatcher. The callback takes a payload as its argument, and then uses a switch statement to determine what actions to take based on the action type listed in the payload. (This action type is the same constant originally defined by the DeLoreanActionCreators way back when.)

// js/stores/DeLoreanStore.js

var AppDispatcher = require('../dispatcher/AppDispatcher');
var ActionTypes = require('../constants/AppConstants').ActionTypes;
var EventEmitter = require('events').EventEmitter;
var assign = require('object-assign');

var CHANGE_EVENT = 'change';

var _speed = 0;

var DeLoreanStore = assign({}, EventEmitter.prototype, {
  emitChange: function() {
    this.emit(CHANGE_EVENT);
  },

  addChangeListener: function(callback) {
    this.on(CHANGE_EVENT, callback);
  },

  removeChangeListener: function(callback) {
    this.removeListener(CHANGE_EVENT, callback);
  },

  getSpeed: function() {
    return _speed;
  }
});

DeLoreanStore.dispatchToken = AppDispatcher.register(function(payload) {
  var action = payload.action;

  switch(action.type) {
  case ActionTypes.ACCELERATE:
    _accelerate(action.mph);
    DeLoreanStore.emitChange();
    break;

  default:
    // noop
  }
});

function _accelerate(mph) {
  _speed += mph;
}

module.exports = DeLoreanStore;

Stores are the one place in a Flux application where state is officially defined. In a simple case, this may just be a running tally of how many times a button was clicked. Based on the action type, the Store’s public methods will utilize private methods to update its privately held data. Finally, it emits a ‘change’ event, and it’s work is done.

Now we’re back in React’s View layer, but not so fast – the state of our application has changed! When the DeLorean component (our Controller-View) receives the new state from the DeLoreanStore, it calls its setState() method. This automatically triggers its render() method, which in turn triggers the render() methods of all of its nested components, and it’s turtles all the way down.

All components in the app now have access to the data in its current state, and await further instructions.

Closing Thoughts

It took me a couple weeks, but I finally feel like I’m understanding Flux, and I really like it! It provides a different way of reasoning about state and data flow in JavaScript applications. React is super straightforward, and as it approaches version 1.0, it is doing what it can to make its API even MORE simple as well as compliant with new ES6 (and even ES7!) features. I’m looking forward to delving into different libraries like RxJS and Immutable.js that take the ideas of functional reactive programming to the next level.

Please check out the full code of my sample app on github!

Resources

Flux Overview – From the official documentation.
Jing Chen’s Original Flux Presentation – There’s some good info about React in here as well, but the Flux part is from 10 to 24 minutes in.
React and Flux: Building Applications with a Unidirectional Data Flow – Good talk by Bill Fisher and Jing Chen where they go through an example chat application in Flux and React.
Flux Chat Example App on Github
Flux: Actions and the Dispatcher – Helpful post by Bill Fisher going into a bit more detail about the Dispatcher, Actions, and Action Creators.

Behind the Scenes of the 'Has Many' Active Record Association

Wed, 08 Oct 2014 20:17:09 +0000

In The Rails 4 Way, Obie Fernandez describes what happens when you “wire up” associations (like has_many and belongs_to) between Active Record models:

When these relationship declarations are executed, Rails uses some metaprogramming magic to dynamically add code to your models. In particular, proxy collection objects are created that let you manipulate the relationship easily.

– The Rails 4 Way, page 181

What exactly is this “metaprogramming magic” that goes on behind the scenes? In this blog post, I hope to uncover exactly what happens when has_many is used in an Active Record model.

TL;DR: When a class inherits from ActiveRecord::Base, it gets access to a large amount of class and instance methods, including the has_many class method. When has_many is called on a class, a GeneratedAssociationMethods module is created on the fly and mixed into the class. A number of instance methods are then defined on this module (these vary depending on the type of association). These methods allow the original class to access and manipulate objects from the associated class passed as an symbol argument to the has_many method. Pretty magical if you ask me.

Diving into the Source Code

The Rails source code has always been intimidating for me. There’s a ton of “metaprogramming magic” going on under the hood that appears incomprehensible or esoteric to a beginner. Though it’s all written in Ruby, which I feel reasonably comfortable with, I’m a total noob when it comes to metaprogramming, so my previous attempts to read the Rails source code have not gone very far.

I try not to feel too bad about this. I’ve only been coding for about six months, and Rails has been constantly developed over the last ten years by some of the best programmers out there.

My first step was to go the source: the Rails source code, that is. I cloned the master branch down to my computer, and started digging in.

git clone git@github.com:rails/rails.git

I started to feel differently about the code once it was on my local machine. Suddenly I felt like I had more control over the code, just as I would with any other Ruby files open in Sublime Text on my computer. Rather than getting distracted by the complexity and sheer magnitude of the Rails codebase, I relied on techniques that I was already confident with and worked my way through from there.

NOTE: I have pinned the links down to a commit in the Rails 4.1.4 era (when this post was written). Plenty of changes have been made since then, so be sure check out the latest version of the repo and investigate the code for yourself!

Starting With What I Know

Let’s make a basic Rails app using Game of Thrones as its domain model. At first, I have two classes, House and Character, both of which inherit from ActiveRecord::Base. I’d wire up the associations between these classes as such:

# app/models/house.rb
class House < ActiveRecord::Base
  has_many :characters
end

# app/models/character.rb
class Character < ActiveRecord::Base
  belongs_to :house
end

This domain logic makes sense; in Game of Thrones, characters often introduce themselves in the fashion “I am Arya of House Stark.” Arya can be said to belong to House Stark, while House Stark can be said to have many characters (a collection, if you will) associated with it.

At this point, here’s what I know about the code in the House model:

The House class inherits from ActiveRecord::Base. This gives the House class and instances of this class access to a number of methods defined in the numerous modules included in the ActiveRecord::Base class. We can now say that the House class is an “Active Record model”, as opposed to a “plain old Ruby object”.
In Rails, an Active Record model is automatically mapped to a table in the database (in this case houses), and attributes (i.e. methods) are created in the model that correspond to each column in this database table. Similarly, each instance of an Active Record model “wraps” one row in this table. This is the key to Rails’s adaptation of the Active Record pattern – database logic is effectively encapsulated into methods on the model layer.
has_many is a “bareword”. In Ruby, barewords can only be local variables, method arguments, keywords (like class, def, and end), and method calls. In this context, it’s neither a local variable nor a method argument, and it’s certainly not a keyword. Therefore it must be a method.
If has_many is a method, then in this context its receiver must be the House class. That makes has_many a class method. It could also be written as self.has_many, but the receiver is implicit here so we don’t need to write self.
It then follows that :characters is a method argument passed to the has_many method. Ruby allows you to leave off the parentheses surrounding method arguments, though it is generally good practice to keep the parentheses in when defining a method with arguments.

Knowing that has_many is a class method defined somewhere in Active Record helped to narrow things down. From here, it was very easy to track the method definition down by doing a global search for def has_many on the codebase using Command + Shift + f in Sublime Text! Only one result appeared – a two-line method in the ActiveRecord::Associations::ClassMethods module:

def has_many(name, scope = nil, options = {}, &extension)
  reflection = Builder::HasMany.build(self, name, scope, options, &extension)
  Reflection.add_reflection self, name, reflection
end

Taking it One Line at a Time

The has_many method signature is actually quite straightforward. Also, there is a copious amount (~200 lines) of documentation directly above the method definition that describes what has_many is, gives examples of how to use it, details the methods that are added when a has_many association is declared, and describes what all of the method arguments it takes are.

Here’s a quick rundown of the arguments that has_many can take:

name: The only required method argument here. The name conventionally takes a symbol the plural form and references a collection embodied by another class. In our case, :characters is passed in as the name parameter. The :characters collection refers to multiple instances of the Character class (or rows of the characters table in the database) that we want to associate with one instance of the House class. This is the one-to-many relationship that lets us know we should use the has_many association to begin with.
scope: Defaults to nil. A scope must be an object with a call method, so lambdas are generally used here. Scopes can help you narrow in on a more targeted set of records to retrieve from the database. In our example, if we only wanted to associate living characters with their houses, we’d use the following scope:

class House < ActiveRecord::Base
  has_many :characters, -> { where(deceased: false) }
end

(Though this might get complicated when dealing with the Iron Islands…)

options: Defaults to an empty hash. Here you can further customize the nature of the association. Some common options to pass here are through: :join_table, dependent: :destroy, polymorphic: true, and foreign_key: :uuid.
&extension: Explicit block argument, as indicated by the ampersand. I have never used this before, but the comments suggest that association extensions are “useful for adding new finders, creators and other factory-type methods to be used as part of the association.”

To figure out the next line, I first tracked down the ActiveRecord::Associations::Builder::HasMany class, which is quite slim, containing just three one-line methods! Unfortunately, build is not one of these methods. I traversed up the inheritance hierarchy, from ActiveRecord::Associations::Builder::HasMany to ActiveRecord::Associations::Builder::CollectionAssociation and finally to ActiveRecord::Associations::Builder::Association, where the class method build is defined.

def self.build(model, name, scope, options, &block)
  if model.dangerous_attribute_method?(name)
    raise ArgumentError, "You tried to define an association named #{name} on the model #{model.name}, but " \
                         "this will conflict with a method #{name} already defined by Active Record. " \
                         "Please choose a different association name."
  end

  builder = create_builder model, name, scope, options, &block
  reflection = builder.build(model)
  define_accessors model, reflection
  define_callbacks model, reflection
  define_validations model, reflection
  builder.define_extensions model
  reflection
end

The ActiveRecord::Associations::Builder::Association::build method takes 5 arguments. The latter four of these arguments are identical to those initially passed to has_many. The first method argument is referred to as model in build’s method signature. Inside the has_many method, the value passed as this argument is self.

Given that the has_many method is inside the ActiveRecord::Associations::ClassMethods module, I assumed that somewhere along the line this module is being mixed into a class. The value of self inside a class method is the class itself – that is, whichever class called the has_many method to begin with (in our case House).

I had a difficult time tracking down where, if anywhere, this module is mixed in to the House class. Then I had a stunning (and obvious) realization: this module is mixed in to every single Active Record model! From here, it was easy to hunt down the offending code in – where else? – ActiveRecord::Base.

module ActiveRecord
  class Base
    # ...
    include Associations
    # ...
  end
  # ...
end

Now we know that in our example, House and :characters are being passed as the model and name arguments to the build method.

Locating the “Metaprogramming Magic”

The build method first checks if the name parameter is “dangerous”, and if so it throws an error. The comment above the dangerous_attribute_name? method states that “a method name is ‘dangerous’ if it is already (re)defined by Active Record, but not by any ancestors. (So ‘puts’ is not dangerous but ‘save’ is.)”

Barring any unforeseen danger, the build method then passes all of its method arguments to the create_builder method, which is called on an implicit receiver. In this context, self is referring to the ActiveRecord::Associations::Builder::HasMany class. The create_builder method checks that the name argument passed to it (:characters) is a symbol, then instantiates a new instance of the ActiveRecord::Associations::Builder::HasMany class. This object is stored as builder, a local variable in the build method above.

Next, the build instance method (different from the build class method) is called on the builder object. The build instance method takes a model (House) as a method argument.

def build(model)
  ActiveRecord::Reflection.create(macro, name, scope, options, model)
end

The only new thing here is macro, which is a method defined on the ActiveRecord::Associations::Builder::HasMany model.

def macro
  :has_many
end

The ActiveRecord::Reflections::create method delegates based on the macro passed to it. In our case, it instantiates an instance of the ActiveRecord::Reflection::HasManyReflection class, which inherits from the ActiveRecord::Reflection::Association class. I’m still not totally clear what reflections in general do, but this bit of documentation was helpful:

Reflection enables interrogating of Active Record classes and objects about their associations and aggregations. This information can, for example, be used in a form builder that takes an Active Record object and creates input fields for all of the attributes depending on their type and displays the associations to other objects.

The value returned is this instance of the ActiveRecord::Reflection::HasManyReflection class, and the original method ActiveRecord::Associations::Builder::HasMany::build class method stores this object to a local variable reflection.

Once I knew what the model and reflection local variables were, it was relatively easy to locate the define_accessors, define_callbacks, define_validations, and define_extensions methods, as they were all in the same file as the build method. In this blog post, I’ll only get into the define_accessors method.

# Defines the setter and getter methods for the association
# class Post < ActiveRecord::Base
#   has_many :comments
# end
#
# Post.first.comments and Post.first.comments= methods are defined by this method...
def self.define_accessors(model, reflection)
  mixin = model.generated_association_methods
  name = reflection.name
  define_readers(mixin, name)
  define_writers(mixin, name)
end

def self.define_readers(mixin, name)
  mixin.class_eval <<-CODE, __FILE__, __LINE__ + 1
    def #{name}(*args)
      association(:#{name}).reader(*args)
    end
  CODE
end

def self.define_writers(mixin, name)
  mixin.class_eval <<-CODE, __FILE__, __LINE__ + 1
    def #{name}=(value)
      association(:#{name}).writer(value)
    end
  CODE
end

This was it. After much searching, I had found the “metaprogramming magic” at the core of Active Record associations.

Interpreting the “Metaprogramming Magic”

Again, I had to start with what I know here. Two method arguments are passed to define_accessors: model and reflection. When the build method calls define_accessors, it uses the same names for these arguments. Back in the build method, model referred to the House Active Record model, and reflection referred to the instance of the ActiveRecord::Reflection::HasManyReflection class.

Moving on to the next line, I found the generated_association_methods method is called on House and the return value is stored in a local variable mixin. I tracked down the method definition of generated_association_methods in the ActiveRecord::Core::ClassMethods module, alongside classics such as find and find_by.

def generated_association_methods
  @generated_association_methods ||= begin
    mod = const_set(:GeneratedAssociationMethods, Module.new)
    include mod
    mod
  end
end

I was really stumped by this one. For one thing, I was distracted by the begin...end block and the PascalCase symbol. I had to take a step back and remember this is just Ruby code! Soon, I realized that this method memoizes the value of a class instance variable @generated_association_methods.

As it turns out, const_set is an instance method on the Module class in the Ruby core library. const_set takes two arguments: the first is a string or symbol that will be the name of the new constant you’re creating, and the second argument is an object that will be set to the value of this constant. The constant is then namespaced under the receiver of the const_set message.

In this case, const_set has an implicit receiver (self). It’s tempting to think that self is the ActiveRecord::Core::ClassMethods module, but as above, this module is actually mixed in to the House class when we have it inherit from ActiveRecord::Base. The receiver here is actually House. Thus, this method creates a new module in the House namespace: House::GeneratedAssociationMethods. (Note that House::GeneratedAssociationMethods is the return value here, but if we were assigning pretty much anything but a new class or module as GeneratedAssociationMethods’s value here, the return value would be that value.)

On the next line of this method, the House::GeneratedAssociationMethods module is included into the House class. This is pretty awesome – I had no idea that you could call include from inside a class method. It totally makes sense though, as the receiver is the class itself, so the use of include here is no different than the typical use of include in the class namespace.

Finally, the House::GeneratedAssociationMethods module is returned, and thus is set as the value of the @generated_association_methods class instance variable. Back in the define_accessors method, this constant is stored as the mixin variable.

Going on to the next line of the define_accessors method, the name message is passed to the reflection variable (our instance of the ActiveRecord::Reflection::HasManyReflection class). This object has an attr_reader for name, which is set upon initialization. If it’s not too far a stretch back to remember, the name attribute here is set to the original name passed as the first argument of the has_many method. A refresher:

class House < ActiveRecord::Base
  has_many :characters
end

The name is simply :characters!

define_accessors delegates to define_readers and define_writers, passing House::GeneratedAssociationMethods and :characters as the mixin and name arguments to both of these methods. For simplicity, I will just focus on the define_readers method.

A Metaprogrammed Method

From here, it is relatively clear to see what happens next. Let’s look at define_readers again:

def self.define_readers(mixin, name)
  mixin.class_eval <<-CODE, __FILE__, __LINE__ + 1
    def #{name}(*args)
      association(:#{name}).reader(*args)
    end
  CODE
end

Aside of a few things in the first line of this method, this actually looks pretty straightforward: we’ve got a new method definition on our hands!

On the first line of the method, class_eval is called on the House::GeneratedAssociationMethods module. class_eval is another instance method of the Module class in Ruby’s core library. It takes a string as an argument, as well as optional parameters for filename and line number. From the documentation:

class_eval(string [, filename [, lineno]]) → obj

Evaluates the string or block in the context of mod, except that when a block is given, constant/class variable lookup is not affected. This can be used to add methods to a class. module_eval returns the result of evaluating its argument. The optional filename and lineno parameters set the text for error messages.

As suspected, class_eval allows us to add methods to a module or class. In this case, we are adding methods to the House::GeneratedAssociationMethods module. As this module has already been included in House, these new methods can be accessed by instances of the House class.

Let’s look at the parameters being passed to this method:

string: The string being passed to the class_eval method is a heredoc denoted by <<-CODE ... CODE. The contents are a dynamic method definition that will look like this in the example we’re working with:

def characters(*args)
  association(:characters).reader(args)
end

filename: I am still not entirely sure what __FILE__ precisely refers to, but I believe it refers to the current file. I am also not very clear about what this means in this context. My educated guess is that it refers to the app/models/house.rb file, but really the method is being added to the House::GeneratedAssociationMethods module, which was created dynamically and doesn’t have a file at all!
lineno: Again, I’m not too clear about this one. The documentation is sparse here, only stating that __LINE__ refers to “The line number, in the current source file, of the current line.”

Almost there! We’ve now got our method definitions, but what do association and reader refer to?

Associations, Caching, and Reflections

It turns out that association is an instance method of the ActiveRecord::Associations module.

def association(name) #:nodoc:
  association = association_instance_get(name)

  if association.nil?
    raise AssociationNotFoundError.new(self, name) unless reflection = self.class._reflect_on_association(name)
    association = reflection.association_class.new(self, reflection)
    association_instance_set(name, association)
  end

  association
end

The association method will pull the association specified by the name argument passed to it (in our case :characters) out of the @association_cache instance variable if it has already been loaded into memory. As defined in the ActiveRecord::Core module, @association_cache is initialized to an empty Hash. Interestingly, this happens whenever you instantiate an Active Record object using the House.new syntax!

If the association is not in the @association_cache (i.e. it hasn’t been loaded into memory), then the association method checks whether the class where the association is defined (House) lists the associated class (Character) as one of its _reflections (a class_attribute which is also initialized to an empty hash). The only way to add a reflection to this hash is through the add_reflection module method. Remember where this is called?

def has_many(name, scope = nil, options = {}, &extension)
  reflection = Builder::HasMany.build(self, name, scope, options, &extension)
  Reflection.add_reflection self, name, reflection
end

In the case of our domain model, the second line of the has_many method will add the key "characters" pointing to the value of our reflection from above – the instance of the ActiveRecord::Reflection::HasManyReflection class – to the _reflections hash.

Let’s make an instance of the House class.

stark = House.create(surname: "Stark", sigil: "Direwolf", motto: "Winter is coming.")

# => #<House id: 1, surname: "Stark", sigil: "Direwolf", motto: "Winter is coming.", created_at: "2014-10-09 20:05:01", updated_at: "2014-10-09 20:05:01">

If we call stark.characters, Rails first checks whether "characters" is a key in House’s _reflections hash. If so, a new object is instantiated based on the type of this reflection – in the case of has_many, it’s an instance of the ActiveRecord::Associations::HasManyAssociation class. The association method then calls association_instance_set, passing :characters and this instance of ActiveRecord::Associations::HasManyAssociation as its parameters. This key value pair is then added to the @association_cache instance variable:

def association_instance_set(name, association)
  @association_cache[name] = association
end

The association local variable is returned from the association method above, and then is sent the reader message, which is defined in the ActiveRecord::Associations::CollectionAssociations class, the superclass of ActiveRecord::Associations::HasManyAssociation.

# Implements the reader method, e.g. foo.items for Foo.has_many :items
def reader(force_reload = false)
  if force_reload
    klass.uncached { reload }
  elsif stale_target?
    reload
  end

  @proxy ||= CollectionProxy.create(klass, self)
end

A Return Value at the End of the Tunnel

Here on the last line of the reader method was @proxy, the final value that is returned by the newly minted characters method. This @proxy instance variable is also the one alluded to by Obie Fernandez when he says “proxy collection objects are created that let you manipulate the relationship easily.”

Here is where it gets interesting: the ActiveRecord::Associations::CollectionProxy class inherits from ActiveRecord::Relation, which is the main class in Rails that deals with database operations. From here, we have to go all the way up the inheritance chain to the ActiveRecord::Delegation::ClassMethods module to find the create method used here.

Now when we call stark.characters, we know without a shadow of a doubt what our return value will be!

stark.characters

# => [#<Character id: 1, name: "Arya", age: 11, catchphrase: "Stick them with the pointy end.", house_id: 1, created_at: "2014-10-09 20:07:27", updated_at: "2014-10-09 20:07:27">]

Hmm. From inside the Rails console, this looks like an array. Let’s check out the class of this collection:

stark.characters.class

# => Character::ActiveRecord_Associations_CollectionProxy

Much better.

Conclusion

This post was a good exercise for me to get more comfortable delving into the Rails source code, to begin to wrap my head around some of the legendary “Rails magic”, and to learn a bit more about metaprogramming in the wild. This is just the tip of the iceberg, but I no longer feel the Rails source code is untouchable. In fact, it’s all just a bunch of great Ruby code!

Resources

Read the Rails source code on Github. It’s terrifying and awesome.
Ruby on Rails API Documentation for ActiveRecord::Associations::ClassMethods
Ruby Core Documentation for Class: Module
The Rails 4 Way
Ruby Tapas: Barewords
class_attributes in Rails

My Favorite Sublime Text Keyboard Shortcuts

Wed, 24 Sep 2014 00:55:44 +0000

Like many software developers, Sublime Text is my preferred text editor. I do almost all of my coding and blogging in Sublime, and as such there are a few convenient keyboard shortcuts I’ve picked up along the way. [Note: All keyboard shortcuts are for the OS X version of Sublime Text 2.02, YMMV.]

1. Command + Enter = New Line Below

Gone are the days of pressing Enter and accidentally cutting a line in half, then having to undo, then having to figure out a way to get the cursor to get to the end of the line before hitting Enter again. This shortcut has saved me a lot of energy – instead of worrying about where the cursor is on a line when hitting Enter, I only need to make sure that the cursor is on that line. Command + Enter takes care of the rest.

2. Command + Shift + Enter = New Line Above

A slight modification of the above, and an insanely useful shortcut. I didn’t realize how much I needed this shortcut until I started using it. It’s perfect if you write a couple of lines at the top of a file (e.g. a Ruby file)

class Dog
end

then realize you need to do something above the first line (e.g. require another file).

require_relative 'cat'

class Dog
end

Zoom up to the top of the page with Command + Up arrow (not a Sublime-specific keyboard shortcut), then hit Command + Shift + Enter, and you’re ready to write.

Another great use case of this for writing Ruby code is when you define a method. I will often instinctually write it like this:

def fetch(bone)
end

and then immediately realize I want to write the body of the method. Command + Shift + Enter has saved me many times here.

3. Command + Shift + d = Duplicate Current Line

Very straightforward and very useful. Simply duplicates the line you call the command on and places the new line below.

4. Control + Shift + k = Delete Current Line

Also straightforward, and a more efficient version of “the number one tool for improving code.”

5. Command + d = Multiple Select of the Same Word

Let’s say I’m refactoring my code and I decide to change the name of a method. I could do a global replacement with Command + Shift + f, but if I want to do a more fine-grained change and have more control over exactly what I’m highlighting and editing, I go with Command + d.

What’s really cool about this is that Sublime gives you a cursor for each highlighted block of text! That means when you change the highlighted text in what you think should be one place, all of the highlighted texts change. This is the first demo on the Sublime Text website – check it out if you haven’t yet!

6. Command + Highlight Text = Multiple Select of Different Words

This one is similar to the Command + d, but is not restricted to highlighting repetitions of the same text. To get this to work, all you have to do is keep pressing the Command key and highlight different regions of text with the trackpad. As with the previous shortcut, this gets really powerful when combined with other keyboard shortcuts, like copy/paste.

In Ruby, you might be setting several attributes of a class to values from a hash. If the keys of the hash happen to be named the same as the keys of the class’s attributes, then there is no need to type these out twice! If you start out with

def import_dog_characteristics(hash)
  self.mood
  self.fur_color
  self.favorite_toy
end

use Command + d to highlight all instances of . in this method, then press the right arrow key so the three cursors are to the left of the attribute’s name. Next, use Alt + Shift + right arrow (not a Sublime-specific keyboard shortcut) to highlight the next word (Command + Shift + right arrow also works because the word is at the end of the line here) and copy it using Command + s. Press right arrow again to get to the cursors to the end of each line, type ` = hash[:` (note that Sublime autocloses the bracket!), and hit Command + v to paste in the attributes.

def import_dog_characteristics(hash)
  self.name = hash[:name]
  self.fur_color = hash[:fur_color]
  self.favorite_toy = hash[:favorite_toy]
end

Now, we could probably do some refactoring here, but that’s another blog post!

7. Command + Shift + p = Set Syntax Highlighting

This shortcut actually opens up the Command Palette – a menu that includes a number of things like package controls and preference settings. Enter “ss” when the search bar pops up to jump down to the various “Set Syntax” options. The search functionality in Sublime Text is great at matching fuzzy queries, so if you just type “ssr” then hit Enter, the text in your file will have Ruby’s syntax highlighting.

This is awesome if you are just sketching out some ideas in a file that you haven’t saved yet (on second thought, you might want to save that file!) or if you’re working in a bin or other file that doesn’t have an extension.

8. Command + t = Find File

Navigation between files can get a bit tricky when in a big project like a Rails app, and it’s easy to fall back on using the mouse to click through folders and open new files. Command + t lets you keep your hands on the keys by opening up a search bar to help you find the file you’re looking for. As with the previous shortcut, Sublime Text has great fuzzy matching for search queries, so you can find the file you’re looking for as easily as possible.

9. Alt + Drag Mouse = Select Rectangle with Multiple Cursors

This one is cool, though admittedly I do not use it as often as the above shortcuts. Sometimes you want to get a cursor in the same column on 10 consecutive lines, or you want to select a 3 x 8 rectangle of text. Alt + drag mouse is the right tool for the job here. For this one to work, be sure that no text on the screen is highlighted, then hold Alt down, and finally drag the mouse over the desired text.

10. Control + Command + Up Arrow = Move Line Up

Great for quickly moving one or more lines up the page. Just highlight the lines you want to move, or make sure the cursor(s) is/are on the line(s) you want to move. Super convenient.

11. Control + Command + Down Arrow = Move Line Down

Same as the last shortcut, but moving in the opposite direction. These two shortcuts have saved me a lot of energy that would have been wasted copying/pasting and using the trackpad to highlight text. Keyboard shortcuts rock!

12. Command + [ = Indent Line by One Tab

Having nice, legible code depends to large extent on the formatting of the code. Though Ruby is not a whitespace-sensitive language, many other programming languages are. In any event, proper indentation is critical to writing legible code, and this a good shortcut to achieving this goal.

13. Command + ] = Unindent Line by One Tab

Same as above, but going the other way. I’ll often use this in combination with Control + Command + down arrow when extracting a method, and these keyboard shortcuts make the process quick and simple!

14. Command + / = Toggle Line Commented/Uncommented

This shortcut comes in handy when testing out new ways to write old code. Instead of deleting the old, working code, simply comment it out, try out the new code, and then, if the new code works, use Control + Shift + k to delete the old code.

Resources

Sublime Text Official Website
Practice Sublime Text Shortcuts
Bonus for Rails developers: SublimeERB. This plugin makes one of the hardest parts of writing a Rails app (writing ERB tags) a piece of cake.

Getting Started With RSpec

Thu, 18 Sep 2014 15:56:23 +0000

RSpec is tool to test Ruby code using a very English-like syntax. RSpec is one of the most popular testing frameworks in the Ruby and Rails communities today, along with minitest (the default testing framework used in Rails 4) and Test::Unit.

Testing is essential to writing reliable code. Not only does it establish a rapid feedback loop, but it also gives you the freedom to refactor your code to your heart’s content while ensuring that the code retains its original functionality.

How to Integrate RSpec Into Your App

It’s very easy to get started with RSpec. To download RSpec run

gem install rspec

In a Ruby app, you can get RSpec up and running with

rspec --init

This will make a .rspec file in the root directory of the application and a spec/ directory containing a spec_helper.rb file.

To get RSpec integrated into a Rails app, add the following code to the Gemfile:

group :development, :test do
  gem 'rspec-rails'
end

and run bundle install to install the RSpec gems in the Rails app. Then run the following command to create the .rspec and spec/spec_helper.rb files:

rails generate rspec:install

Because Rails autogenerates a test suite through the rails new command, it can also be helpful to tell Rails to not include these files with the -T flag when creating a new app.

rails new greatest_app_ever -T

Now that RSpec is installed, you can run your specs from the command line with rspec.

$ rspec
No examples found.

Finished in 0.00037 seconds (files took 0.11253 seconds to load)
0 examples, 0 failures

Structuring a Spec File

Let’s say I want to write a basic Ruby app that will keep track of some different espresso-based beverages. Here’s the file directory structure after running rspec --init:

espresso-maker/
|
|__ lib/
|   |
|   |__ espresso.rb
|
|__ spec/
|   |
|   |__ spec_helper.rb
|
|__ .rspec

So far these files are pretty empty. Here’s espresso.rb:

class Espresso
end

I want to take a test-first approach to this app, so I’ll go ahead and set up a spec file to test the Espresso class. Any file that ends in _spec.rb will be run by the rspec command, but it is best to name the spec files in the model_name_spec.rb format. In the spec directory, I made an espresso_spec.rb file to house all the tests I’ll write for the Espresso model.

require_relative 'spec_helper'

describe Espresso do
end

At the top of the espresso_spec.rb file, I required the spec_helper.rb file, where I will list all of the file dependencies so that they don’t clutter up this spec file. Then I opened a describe block for the Espresso class - all of the tests in this file should go inside this block. Note that it’s OK to use the constant Espresso in this case, but generally you will pass in a string as an argument to the RSpec methods describe, context, and it.

In order for the tests to run properly, I also need to add the following line to the spec/spec_helper.rb file:

require_relative '../lib/espresso'

Without this require_relative statement, RSpec will not understand what Espresso is and will throw an uninitialized constant NameError.

describe Espresso do
  describe '#name' do
  end

  describe '#ingredients' do
  end

  describe '.country_of_origin' do
  end
end

Inside the describe Espresso block, I opened a describe block for each public method that will be part of Espresso’s API. These method-specific describe blocks should use the #method_name notation for instance methods and the .method_name notation for class methods.

describe Espresso do
  describe '#name' do
    it 'is espresso' do
    end
  end

  describe '#ingredients' do
    it 'one part espresso' do
    end
  end

  describe '.country_of_origin' do
    it 'is from Kenya' do
    end
  end
end

Within each describe block, I can open it blocks that will contain the actual expectations to be tested. it takes a terse (less than 40 characters) description of the specific expectation I’m setting on the method for this test as an argument.

Inside of it blocks, the RSpec 3 syntax uses the expect method to set up an expectation.

expect(1 + 1).to eq(2)

In general, you pass an expression to be evaluated to the expect method and chain to to the end of this method. Then you call a matcher method and pass in the value you expect the argument passed to expect to evaluate to. Some of the most common RSpec matcher methods are eq, include, match, and respond_to, all of which closely correspond to Ruby methods.

expect([1, 2, 3]).to include(3)
expect("Hello world, my name is Chris!").to match(/Hello world/)
expect("an ordinary string").to respond_to(:upcase)

Writing the Specs

Now it’s time to write our first test!

describe Espresso do
  describe '#name' do
    it 'is espresso' do
      espresso = Espresso.new
      expect(espresso.name).to eq("espresso")
    end
  end
end

Here we’re instantiating a new Espresso object and setting up an expectation that the #name method will return the value “espresso” when it is sent to an instance of the Espresso class. We run rspec and the test fails.

$ rspec

Espresso
  #name
    is espresso (FAILED - 1)

Failures:

  1) Espresso#name is espresso
     Failure/Error: expect(espresso.name).to eq("espresso")
     NoMethodError:
       undefined method `name' for #<Espresso:0x00000103131178>
     # ./spec/espresso_spec.rb:7:in `block (3 levels) in <top (required)>'

Finished in 0.00057 seconds (files took 0.11633 seconds to load)
1 example, 1 failure

Failed examples:

rspec ./spec/espresso_spec.rb:5 # Espresso#name is espresso

Now the cycle of “Red, Green, Refactor” can begin. We have a failing (red) test, now let’s do the minimum possible work to get the test to pass (green). Once we have green tests, then we can mercilessly refactor while ensuring the tests stay green, and thus the functionality of the code stays the same.

Based on the error message above, I know the first step should be adding a name method to the Espresso class.

class Espresso
  def name
  end
end

Running rspec again gives a bit more guidance about what the name method should return:

Failures:

  1) Espresso#name is espresso
     Failure/Error: expect(espresso.name).to eq("espresso")
       
       expected: "espresso"
            got: nil
       
       (compared using ==)

Let’s make the test go green!

class Espresso
  def name
    "espresso"
  end
end

$ rspec

Espresso
  #name
    is espresso

Finished in 0.00158 seconds (files took 0.12372 seconds to load)
1 example, 0 failures

Success!

Introducing `context`s

Now I want to have the #name method return different results depending on whether or not the espresso has milk in it. I could just write a new it '...' expectation inside of the describe "#name" block, but this might get unwieldy in short order. Namespacing is important to writing readable, easy-to-follow tests. If there are many facets of a method to be tested, we can group the nested it blocks inside of context blocks.

describe Espresso do

  describe '#name' do
    context 'without milk' do
      it 'is espresso' do
        espresso = Espresso.new
        expect(espresso.name).to eq("espresso")
      end
    end

    context 'with milk' do
      it 'is macchiato' do
        espresso = Espresso.new
        espresso.add_milk
        expect(espresso.name).to eq("macchiato")
      end
    end
  end
end

Here I’ve written two context blocks, one for an espresso with milk and one for an espresso without milk. In order to get the milk into the espresso, I’ve also included an add_milk method – this should also be tested as part of Espresso’s API. Before getting the next test to pass, I fleshed out the Espresso class a bit to support this new method.

class Espresso
  def initialize
    @milk = false
  end

  def name
    "espresso"
  end

  def add_milk
    self.milk = true
  end

  private
    attr_accessor :milk
    def has_milk?
      milk
    end
end

The RSpec failure for the second test then read:

Failures:

  1) Espresso#name with milk is macchiato
     Failure/Error: expect(espresso.name).to eq("macchiato")
       
       expected: "macchiato"
            got: "espresso"
       
       (compared using ==)

In order to get this test to pass, I implemented the simplest change possible to the name method:

def name
  has_milk? ? "macchiato" : "espresso"
end

then ran rspec and enjoyed the nicely namespaced RSpec output:

$ rspec

Espresso
  #name
    without milk
      is espresso
    with milk
      is macchiato

Finished in 0.00147 seconds (files took 0.12149 seconds to load)
2 examples, 0 failures

Going forward, I could also add contexts for things like “without ice” and “with ice”.

Next Steps

This is a basic introduction to writing the first few tests of a Ruby app with RSpec, and is far from a complete resource. Already we can see that there is some repetition in the tests above (espresso = Espresso.new) that should be abstracted out. RSpec has a number of convenient methods to help you DRY out your tests, such as let, before, subject, and it_behaves_like. There’s also clearly some more refactoring that can be done on the code itself – but now we have a safety net to fall back on when we start refactoring.

For more comprehensive info about RSpec, check out the resources below!

Resources

Official RSpec Documentation
Better Specs - A collection of RSpec best practices
Testing with RSpec on Code School - Still uses the RSpec 2 should syntax, but otherwise a great intro to RSpec.

China Map Project - Part 4: Introducing Polymorphism

Sat, 13 Sep 2014 21:33:42 +0000

This the fourth and final post in a series about my recent side project, A Map of China. Check out the first post, second post, third post, and the project repo on Github.

The Road to Polymorphism

When I started this project, I set out to make an interactive map that displayed a variety of data about the Chinese provinces. But this very premise – mapping Chinese provinces – turned out to be far more nuanced and complex than I realized at the outset.

Technically speaking, China has 22 provinces (e.g. Shaanxi, Guangdong), 5 autonomous regions (e.g. Tibet, Inner Mongolia), 4 municipalities (Beijing, Shanghai, Chongqing, and Tianjin), and 2 special administrative regions (SARs) (Hong Kong and Macau). [Note: Any issues pertaining to Taiwan fall outside the scope of this project.]

Originally, I had named the main model in my Rails app Province, but later decided that Region would be more suitable. I considered breaking up the Region model into several smaller models (i.e. Province, AutonomousRegion, etc.), but I wanted the data output from the JSON API to all be standardized and accessible from one endpoint, so I kept all regions contained in one model.

The issue of polymorphism came up when I realized that I was using a large number of conditional checks on the region’s name in my Wikipedia scraping code. The most prominent among these was the following:

if %w{ Hong\ Kong Macau }.include?(region.name)
  # ...
else
  # ...
end

While I didn’t have a SpecialAdministrativeRegion model, this conditional logic more or less amounted to a type check that significantly bogged down the readability and clarity of my code.

Recently I have been reading through Refactoring: Ruby Edition, and found a refactoring that suited my needs in this case: Replace Type Code with Polymorphism.

Extract Method

Before I was able to implement the Replace Type Code with Polymorphism refactoring, I needed to continue refactoring the lengthy compute method in the RegionAssembler class from the last post. Within the compute method, I had been assigning about ten different pieces of data to the Region objects and saving them to the database. In order to preserve the compute API set up in the last post, I had compute delegate to a number of smaller methods, which I removed from compute using Extract Method.

Quoting Martin Fowler:

Extract Method is one of the most common refactorings I do. I look at a method that is too long or look at code that needs a comment to understand its purpose. I then turn that fragment of code into its own method.

I prefer short, well-named methods for several reasons. First, it increases the chances that other methods can use a method when the method is finely grained. Second, it allows the higher-level methods to read more like a series of comments. Overriding also is easier when the methods are finely grained.

– Refactoring: Ruby Edition, page 102

Extract Method is an incredibly useful and easy refactoring to undertake. The mechanics are as follows:

1. Create a new method, and name it after the intention of the method (name it by what it does, not by how it does it).
2. Copy the extracted code from the source method into the new target method.

Steps 3-6 deal with handling local variables, which I already extracted out of compute in the Replace Method with Method Object refactoring in the last post.

7. Replace the extracted code in the source method with a call to the target method.
8. Test.
-- Refactoring: Ruby Edition, page 103

In practice, I looked for chunks of code in the compute method like this:

def compute
  # ...
  if %w{ Hong\ Kong Macau }.include?(region.name)
    region.gdp_per_capita = gdp_per_cap[1].gsub(',', '').to_i
  else
    region.gdp_per_capita = gdp_per_cap[3].gsub(',', '').to_i
  end
  # ...
end

then I copied the code out into its own method:

def assign_gdp_per_cap
  if %w{ Hong\ Kong Macau }.include?(region.name)
    region.gdp_per_capita = gdp_per_cap[1].gsub(',', '').to_i
  else
    region.gdp_per_capita = gdp_per_cap[3].gsub(',', '').to_i
  end
end

and replaced the original chunk of code with a call to the extracted method:

def compute
  # ...
  assign_gdp_per_cap
  # ...
end

I methodically went through the compute method and used Extract Method until all logical units of work had been extracted. In the end, the compute method had shrunk from almost 100 lines into this succinct delegator method:

def compute
  assign_territorial_designation
  assign_lat_lng
  assign_capital
  assign_area_info
  assign_gdp_usd
  assign_gdp_cny
  assign_gdp_per_cap
  assign_jvector_code
  region.save
end

Replace Type Code with Polymorphism

With the RegionAssembler class now broken down into small methods, it was much easier to see how to implement the Replace Type Code with Polymorphism refactoring. Here are the mechanics for carrying out this refactoring:

Create a class to represent each type code variant.

Change the class that uses the type code into a module. Include the module into each of the new type classes.

Change the callers of the original class to create an instance of the desired type instead.

Test.

Choose one of the methods that use the type code. Override the method on one of the type classes.

Test.

Do the same for the other type classes, removing the method on the module when you’re done.

Test.

Repeat for the other methods that use the type code.

Test.

Remove the module if it no longer houses useful behavior.

– Refactoring: Ruby Edition, page 226-227

In practice my execution was also a bit similar to the Replace Conditional with Polymorphism refactoring (Refactoring: Ruby Edition, page 279-284).

First I made classes for each regional classification, turned the RegionAssembler class into a module, and included the RegionAssembler module in each class.

module RegionAssembler
  # ...
end

class ProvinceAssembler
  include RegionAssembler
end

class AutonomousRegionAssembler
  include RegionAssembler
end

class MunicipalityAssembler
  include RegionAssembler
end

class SARAssembler
  include RegionAssembler
end

Next I modified the scrape_all_regions method in the ChinaScraper class to appropriately call each of these new classes.

def scrape_all_regions
  Region.all.each do |region|
    page = Nokogiri::HTML(open(region.url))
    case region.name
    when "Hong Kong", "Macau"
      SARAssembler.new(region, page).compute
    when "Beijing", "Chongqing", "Shanghai", "Tianjin"
      MunicipalityAssembler.new(region, page).compute
    when "Guangxi", "Inner Mongolia", "Ningxia", "Xinjiang", "Tibet"
      AutonomousRegionAssembler.new(region, page).compute
    else
      ProvinceAssembler.new(region, page).compute
    end 
  end
end

Then, one at a time, I began overriding the methods from the RegionAssembler module in individual classes. I started doing this in the SARAssembler class first because the Hong Kong and Macau Wikipedia pages had the most special cases that led to the overuse of conditionals in the first place.

In practice, here’s how the overriding worked. I started with the assign_territorial_designation method (extracted out of the compute method above) in the RegionAssembler module:

def assign_territorial_designation
  if %w{ Hong\ Kong Macau }.include?(region.name)
    region.territorial_designation = page.search("tr td a").find { |a| a.text.match(/special/i) }.text.split(" of ").first
  else
    region.territorial_designation = page.search("span.category a").text
  end

  region.territorial_designation = region.territorial_designation.split(' ').map(&:capitalize).join(' ')
end

I copied this method into the SARAssembler class, removed all but one line, and slightly refactored the text selector.

def assign_territorial_designation
  region.territorial_designation = page.search("tr td a").
                                     find { |a| a.text.match(/special/i) }.
                                     attributes["title"].value.
                                     split(' ').map(&:capitalize).join(' ')
end

Then in the RegionAssembler module, I was able to turn this method into a nice one-liner.

def assign_territorial_designation
  region.territorial_designation = page.search("span.category a").text.
                                     split(' ').map(&:capitalize).join(' ')
end

At this point, it became apparent that this method actually had two responsibilities: assigning the accurate text and making sure it was in title caps. There was a clear opportunity to extract a title_caps method:

def title_caps(string)
  string.split(' ').map(&:capitalize).join(' ')
end

I could then further simplify the assign_territorial_designation method as such:

def assign_territorial_designation
  region.territorial_designation = title_caps(page.search("span.category a").text)
end

This refactoring was particularly helpful for methods that applied to certain classes but not to others. Take for instance the assign_capital method. Municipalities and SARs are cities, and therefore don’t have capitals. This method clearly doesn’t apply to them. Yet the original assign_capital method spent a significant amount of effort checking whether or not it applied.

def assign_capital
  if !%w{ Beijing Chongqing Shanghai Tianjin Hong\ Kong Macau }.include?(region.name)
    region.capital = page.search("tr.mergedtoprow a")[0].text
  end
end

In the SARAssembler and MunicipalityAssembler classes, I overwrote this with an empty method:

def assign_capital
end

and then was able to refactor the assign_capital method in the module into a much simpler version:

def assign_capital
  region.capital = page.search("tr.mergedtoprow a")[0].text
end

In the end, I kept the RegionAssembler module, as there would have been too much code duplication if I copied over all of the methods into each of the four classes. At 80 lines, the RegionAssembler module is not exactly small, but the majority of its methods are one-liners, and the type checking is gone.

The Final Step (For Now…)

Starting from one monolithic ChinaScraper class, I used Replace Method with Method Object to extract out most of the logic from this class into a RegionScraper class. Then I went to town using Extract Method to create a bunch of small methods that followed the Single Responsibility Principle (SRP). Having done this, it was clear that my code could benefit from a Replace Type Code with Polymorphism refactoring.

All told, the shape of my code morphed from its original form – one 170 line class hinging on a 90 line method – to a more modular form where code was sectioned off into five classes and two modules, and methods were used for one purpose only.

The final step here was to take all of this code out of the db/seeds.rb file and put it into the app/models directory. This directory’s file structure now looks like this:

app/
|
|__ models/
    |
    |__ concerns/
    |   |
    |   |__ j_vectorable.rb
    |   |
    |   |__ region_assembler.rb
    |
    |__ autonomous_region_assembler.rb
    |
    |__ china_scraper.rb
    |
    |__ municipality_assembler.rb
    |
    |__ province_assembler.rb
    |
    |__ region.rb
    |
    |__ sar_assembler.rb

And the final db/seeds.rb file is now nice and succinct:

ChinaScraper.new.run

Room for Improvement

This was a fun project and I really enjoyed taking the time to refactor my code and make it much more clear, concise, and easy to expand upon going forward.

There are a few areas in which I think I can improve upon in this and future projects:

Regular Expressions. I think I got better at writing regular expressions in this project, but it is possible that I would not have needed the polymorphic refactoring if I had used more accurate regular expressions. Regexps are incredibly powerful, and I want to learn a lot more about them.
Testing. You may have noticed that I did not follow one very important step of the refactorings from Martin Fowler’s book: “Test.” I am a bit ashamed to admit that I do not have any test coverage on this app. I am not sure whether/how to test the entire Nokogiri and web scraping aspect of the project. This is something I have to look into more for this specific use case, and in general I need to gain more experience doing test-first development.
Working more with maps. I am glad that I was able to get the China map up and running, and now I am excited to try working with other third-party libraries to create more interesting data visualizations!

China Map Project - Part 3: Refactoring Using Replace Method with Method Object

Fri, 12 Sep 2014 19:05:42 +0000

This post is the third in a series about my recent side project, A Map of China. Check out the first post, second post, and the project repo on Github.

Refactoring!

I enjoy refactoring. I’ve watched a few great technical talks on refactoring and delved further into the topic with two amazing books: Practical Object-Oriented Design in Ruby by Sandi Metz and Refactoring: Ruby Edition by Jay Fields, Shane Harvie, and Martin Fowler (with Kent Beck).

One main lesson I’ve gleaned from these resources and applying their recommendations to my own code is the importance of having clean, succinct, well-designed code. Code that is easy to read is easy to understand and easy to change when the time comes. I find that pushing my code further towards this goal is both highly challenging and highly rewarding work.

Recently, I have been reading through Refactoring: Ruby Edition and trying to implement some of the refactoring patterns it outlines into my own code. With regards to my China map application, one pattern struck me as particularly useful: Replace Method with Method Object. Quoting Martin Fowler:

In this book I emphasize the beauty of small methods. By extracting pieces out of a large method, you make things much more comprehensible.

The difficulty in decomposing a method lies in local variables. If they are rampant, decomposition can be difficult. Using Replace Temp with Query helps to reduce this burden, but occasionally you may find you cannot break down a method that needs breaking. In this case you reach deep into the tool bag and get out your Method Object.

– Refactoring: Ruby Edition, page 128

The mechanics of this refactoring (which was “stolen shamelessly from Kent Beck’s Smalltalk Best Practice Patterns”) are as follows:

Create a new class, name it after the method.

Give the new class an attribute for the object that hosted the original method (the source object) and an attribute for each temporary variable and each parameter in the method.

Give the new class a constructor that takes the source object and each parameter.

Give the new class a method named “compute”

Copy the body of the original method into compute. Use the source object instance variable for any invocations of the methods on the original object.

Test.

Replace the old method with one that creates the new object and calls compute.

– Refactoring: Ruby Edition, page 129

Seeing the Opportunity for Refactoring

In my original db/seeds.rb file, I had one ChinaScraper class that handled… everything. Even though this was a simple app, that is still a red flag. Before refactoring, the ChinaScraper class was just shy of 170 lines of code – another big red flag. Within that class, there was one 94 line method (!!!) – scrape_all_regions – doing the bulk of the work. It scraped each region’s Wikipedia page, then went through and assigned about ten different attributes to the region, then saved the region and its various attributes to the database.

def scrape_all_regions
  Region.all.each do |region|
    page = Nokogiri::HTML(open(region.url))

    if %w{ Hong\ Kong Macau }.include?(region.name)
      region.territorial_designation = page.search("tr td a").find {|a| a.text.match(/special/i) }.text.split(" of ").first
    else
      region.territorial_designation = page.search("span.category a").text
    end

    region.territorial_designation = region.territorial_designation.split(' ').map(&:capitalize).join(' ')

    # ... 80 more lines of code ...

    region.save
  end
end

As you can begin to see in this snippet, the scrape_all_regions method contained many conditional statements that essentially checked a region’s territorial_designation. This type checking was inevitable given that the HTML/CSS content of the individual region pages varied significantly based on their territorial designations – more on this in the next post.

It was clear that the scrape_all_regions method had far too many (i.e. more than one) responsibilities and several local variables, so I reached deep into the tool bag for the Replace Method with Method Object refactoring.

Implementing Replace Method with Method Object

First, I made a new RegionAssembler class (a slightly more accurate name than the original method) and assigned attributes for all parameters in the scrape_all_regions method:

class RegionAssembler
  attr_reader :region, :page
end

The region ActiveRecord object and page Nokogiri object would have to be passed through to a new instance of the RegionAssembler class upon initialization.

def initialize(region, page)
  @region, @page = region, page
end

The local variables from the original scrape_all_regions method were slightly more complicated, so instead of assigning them to attr_readers, I used Extract Method for the three local variables in scrape_all_regions and placed the extracted methods in the RegionAssembler class.

def area_info
  @area_info ||= page.search("tr.mergedrow").select {|t| t.text.match(/km2/i) }
end

def monetary_info
  if %w{ Beijing Chongqing }.include?(region.name)
    @monetary_info ||= page.search("tr.mergedrow td").find {|tr| tr.text.match(/cny/i) }.text.split(/\s| /)
  elsif %w{ Shanghai Tianjin }.include?(region.name)
    @monetary_info ||= page.search("tr.mergedrow td").find {|tr| tr.text.match(/cny/i) }.text.split(/\s| |cny|usd|\$/i)
  elsif %w{ Guangdong }.include?(region.name)
    @monetary_info ||= page.search("tr.mergedtoprow td").find {|tr| tr.text.match(/cny/i) }.text.split(/\s| |\$/i)
  elsif %w{ Hong\ Kong Macau }.include?(region.name)
    @monetary_info ||= page.search("tr.mergedrow td").select {|tr| tr.text.match(/\$/) }[1].text.split(/\s|\$/)
  else
    @monetary_info ||= page.search("tr.mergedtoprow td").find {|tr| tr.text.match(/cny/i) }.text.split(' ')
  end
end

def gdp_per_cap
  if %w{ Guangdong Hubei }.include?(region.name)
    @gdp_per_cap ||= page.search("tr.mergedrow td").find {|tr| tr.text.match(/cny/i) }.text.split(/\s|\$/)
  elsif %w{ Shanghai }.include?(region.name)
    @gdp_per_cap ||= page.search("tr.mergedrow td").select {|tr| tr.text.match(/cny/i) }.last.text.split(/\s|\$|US/)
  elsif %w{ Tianjin }.include?(region.name)
    @gdp_per_cap ||= page.search("tr.mergedrow td").select {|tr| tr.text.match(/cny/i) }.last.text.split(/\s|\)/)
  elsif %w{ Hong\ Kong Macau }.include?(region.name)
    @gdp_per_cap ||= page.search("tr.mergedbottomrow td").select {|tr| tr.text.match(/\$/) }.last.text.split(/\s|\$|\[/)
  else
    @gdp_per_cap ||= page.search("tr.mergedrow td").select {|tr| tr.text.match(/cny/i) }.last.text.split(' ')
  end
end

Next, I moved the remainder of the scrape_all_regions method from the ChinaScraper class to the RegionAssembler class and renamed it as compute in the RegionAssembler class.

def compute
  if %w{ Hong\ Kong Macau }.include?(region.name)
    region.territorial_designation = page.search("tr td a").find {|a| a.text.match(/special/i) }.text.split(" of ").first
  else
    region.territorial_designation = page.search("span.category a").text
  end

  region.territorial_designation = region.territorial_designation.split(' ').map(&:capitalize).join(' ')
  
  # ... 50 more lines of code ...

  region.save
end

Finally, from inside the ChinaScraper class, I instantiated new RegionAssembler objects and delegated the heavy lifting to them by sending them a simple message: compute.

def scrape_all_regions
  Region.all.each do |region|
    page = Nokogiri::HTML(open(region.url))
    RegionAssembler.new(region, page).compute
  end
end

Awesome four line method! The ChinaScraper class is now a relatively slim 36 lines of code and adheres much more to the Single Responsibility Principle (SRP) than its previous 170 line incarnation.

Closing Thoughts

This initial refactoring made all subsequent refactorings to the db/seeds.rb file significantly easier. It seems like a pretty minimal refactoring at this point, but replacing the scrape_all_regions method with a class was a great first step of breaking the problem up into smaller pieces. This allowed me to think more freely about the problem and began the process of reducing the clutter in my code.

As I alluded to above, the regional classifications of China that led to a glut of conditional statements were also an open door to a polymorphic refactoring, which is the topic of the next (and last!) post in this series.

Chris Callahan

React.Children and the React Top-Level API

React.Children.map

React.Children.forEach

React.Children.count

React.Children.only

React.Children.toArray

Conclusion

Debugging CouchDB Design Documents

How to Configure CouchDB

“console.log Debugging” Inside CouchDB

Wrapping up

Resources for Learning React & Flux

Why React?

React Basics

Flux

React Router

Server-Side Rendering

Tutorials

Cool React Projects

Official Documentation

Read the Source Code

Array Iteration Methods in JavaScript

The Big Three: map(), filter() & reduce()

Array.prototype.map()

Array.prototype.filter()

Array.prototype.reduce()

Friends of reduce()

Array.prototype.reduceRight()

Array.prototype.every()

Array.prototype.some()

A More Expressive for Idiom?

Array.prototype.forEach()

Conclusion

Resources

88 mph With Flux and React

Conceptual Model

Architecture

Writing the App… The Flux Way

Closing Thoughts

Resources

Behind the Scenes of the 'Has Many' Active Record Association

Diving into the Source Code

Starting With What I Know

Taking it One Line at a Time

Locating the “Metaprogramming Magic”

Interpreting the “Metaprogramming Magic”

A Metaprogrammed Method

Associations, Caching, and Reflections

A Return Value at the End of the Tunnel

Conclusion

Resources

My Favorite Sublime Text Keyboard Shortcuts

1. Command + Enter = New Line Below

2. Command + Shift + Enter = New Line Above

3. Command + Shift + d = Duplicate Current Line

4. Control + Shift + k = Delete Current Line

5. Command + d = Multiple Select of the Same Word

6. Command + Highlight Text = Multiple Select of Different Words

7. Command + Shift + p = Set Syntax Highlighting

8. Command + t = Find File

9. Alt + Drag Mouse = Select Rectangle with Multiple Cursors

10. Control + Command + Up Arrow = Move Line Up

11. Control + Command + Down Arrow = Move Line Down

12. Command + [ = Indent Line by One Tab

13. Command + ] = Unindent Line by One Tab

14. Command + / = Toggle Line Commented/Uncommented

Resources

Getting Started With RSpec

How to Integrate RSpec Into Your App

Structuring a Spec File

Writing the Specs

Introducing contexts

Next Steps

Resources

China Map Project - Part 4: Introducing Polymorphism

The Road to Polymorphism

Extract Method

Replace Type Code with Polymorphism

The Final Step (For Now…)

`React.Children.map`

`React.Children.forEach`

`React.Children.count`

`React.Children.only`

`React.Children.toArray`

“`console.log` Debugging” Inside CouchDB

The Big Three: `map()`, `filter()` & `reduce()`

`Array.prototype.map()`

`Array.prototype.filter()`

`Array.prototype.reduce()`

Friends of `reduce()`

`Array.prototype.reduceRight()`

`Array.prototype.every()`

`Array.prototype.some()`

A More Expressive `for` Idiom?

`Array.prototype.forEach()`

Introducing `context`s