Readme, methods & Spikes

Summary

  1. Readme file.
  2. Methods
  3. Spikes

Session

I didn’t do much last week (at least it felt like it), I fixed the two macbooks and I had to do small “hacking”, I discovered about daemons and launchd.plist in my research to fix the odd behavior I was having.

I found SleepWatcher by Bernhard Baer

SleepWatcher is a command line tool (daemon) that monitors sleep, wakeup and idleness of a Mac, It can be used to execute a Unix command when the Mac or the display of the Mac goes to sleep mode or wakes up, after a given time without user interaction.

I had a problem with the screen of the MacBook I am using and I couldn’t make it go to sleep, so I found the command pmset sleepnow that sends directly to sleep the MacBook and this was working, so with the SleepWatcher tool, I was able to trigger this command as soon as the screen was going to sleep.

So I don’t have a full working MacBook but from two broken MacBooks I was able to get one working, in terms of performance is working nice, the only thing is that the display sleep doesn’t work, but with this I fixed.

1. Readme file

AirCrewMentor took a look at the Readme file I had little improved and started to correct it, since it wasn’t really useful at all.

So here is the structure we are using on our Readme file:

  • Mortgage Calculator.
    • Installation
    • Usage
      • On Node.js
      • On the browser
    • Methods.
      • MortgageCalculator.getMaxMortgage
        • Options
      • MortgageCalculator.formatMoney
        • Number
        • Options
    • Contributor notes:
    • Notes:

AirCrewMentor when is going into a personal project, he uses the Readme as a guideline, he usually starts with it and then goes to code what he described.

James Shore uses a to-do list to go ahead with the project and with every commit deletes the task he has completed.

I think each method has its advantages, with AirCrewMentor’s method you keep focused on the main task, and you have the advantage that the documentation is going to be really solid since it is the guideline you used to commit the project. As a con I would say that it is harder to focus on smaller tasks.

With James method I think it is really easy to add new tasks and to focus only in small tasks, but then you’ll have to generate the Readme from scratch once the project is done, and it will require an effort to make it as solid as the one made with AirCrewMentor’s method.

I personally think AirCrewMentor’s is more useful but as my low level in coding it is really challenging. Of course I need more coding to get a valuable opinion on this.


With AirCrewMentor’s method we were defining the variables we were going to use and creating new ones that the methods should have. This way we encourage the readability of our code and give it more sense. So when a new user needs to use it, reading the code could be enough.

2. Methods

Today I learned a basic stuff that haunted me for some time but never had the time to look into it.

So I am coding every function into a separate file, well, not every function but the ones that are related or the ones that are going to be a method of our module. Always respecting the node.js way

But I really didn’t knew exactly why (totally), now I know a little more.

On our index page I created the following code:

module.exports = {
  formatMoney: require("./format_money.js")
};

So now, we can use the method mortgageCalculator.formatMoney(Number, options); in our code.

3. Spikes

So we created a spike to test that our method was working the way we wanted.

// this spike is going to validate the public interface
// described on the readme file.
var mortgageCalculator = require("../");
var options = {
  symbol: "€"
};
var result = mortgageCalculator.formatMoney(123456.9998, options);
console.log(result);

So here we can see that we didn’t specify the location of our file, this is because node.js looks for its location into package.json specified into the "main": "lib/index.js"

Then we only have to call it with the following formula:

$ node spikes/name_of_the_file.js

Homework

  • Hands on coding!!
  • Get the code ready to publish on npm.
Thanks for your time @aircrewmentor, you are a true friend. (I already knew it, but know you are over the charts)

TDD, clone & Readme

Summary

  1. Status of the project and TDD applied.
  2. _.clone
  3. Readme

Session

So, after the holidays AirCrewMentor had, it was time to get our hands back on work.

I subscribed to Let’s code Javascript to learn the technique of James Shore with the TDD style and I found it really useful.

AirCrewMentor was already pressing me to use this sort of Workflow and I have to say that now I can see why was he so persistent, it is a really good way of code that guarantee that your code is delivered well tested and free of errors.

So James Shore has the following workflow (basically speaking):

  • Check lint (if there are errors the test are not performed, they need to be fixed first)
  • Test the code.

James uses Jake for his automation tool that includes every process to do.

I am feeling comfortable with Make so I decided to adapt the TDD with it.

Here you can see the makefile I am actually using:

SHELL = /bin/bash
MAKEFLAGS += --no-print-directory --silent
export PATH := ./node_modules/.bin:$(PATH):./bin
LINT_DIR = $(wildcard *.js test/*.js lib/*.js test/**/*.json spikes/*)

setup:
  npm install

grunt-server:
  grunt server

path:
  echo $$PATH

which:
  which grunt

# run test with dependencies lint and jscs
test: setup lint style
  echo "Test started"
  mocha test/

# Dev mode for continuous testing
dev:
  mocha --watch test

lint:
  echo "Linting started..."
  jshint $(LINT_DIR)
  echo "Linting finished without errors"
  echo

style:
  echo "Checking style..."
  jscs $(LINT_DIR)
  echo

# Continuous Integration Test Runner
ci: lint style test
  echo "1. Make sure 'git status' is clean."
  echo "2. Build on the integration box."
  echo "3. 'git checkout integration'"
  echo "4. 'git merge branch_desired_to_bring_to_integration --no-ff --log'"
  echo "5. 'git checkout master'"

.PHONY: test

I’ve discovered JSCS, a tool I didn’t know it was there, so I decided to add it to my workflow. It has been a good discovery for me, a pain in the ass sometimes but my code-style is better, so I am going to use it.

1. Status of the project and TDD applied.

So, I showed AirCrewMentor the project I created, with the workflow and how I used to improve the formatMoney function I had.

To see all the test passing was a blessing, as well as the code well checked without lint or style errors.

I have to admit that I am proud of what I’ve done, how I am using constants into the Makefile to easier everything, how the workflow is really flowing and doing everything we want to, AirCrewMentor taught me since the beginning how to do it, but now I am the one using it at my wish, I love it, I know it is not a big thing but it is a thing I’ve done.

AirCrewMentor loved the work I’ve done and how I have been able to adapt and learn the TDD technique, even so I still have a lot to learn ahead of me.

I am really happy he liked it that much.

2. _.clone

So, viewing the code, we went over my way of assigning the default values, I wasn’t respecting the original options object passed to the function, so we created the following test to be sure of it:

  it("should not modify original options", function () {
    var options = {
      "places": 4
    };
    formatMoney(2000, options);
    expect(options).to.eql({"places": 4});
  });

And it was failing.😦

So we saw the _.clone method from Lodash, that could be implemented the following way:

options = _.defaults(_.clone(options || {}), defaultOptions);

but we saw many ways of doing the same, we finally went and created our own method:

  clone = _.compose(JSON.parse, JSON.stringify);

Now the test was passing🙂

In the following Screenshot of node.js we can see how our method clone will work.

Screenshot 2015-07-11 19.49.32

3. Readme

AirCrewMentor explained how important is the Readme file in our project and how he uses it as a guideline of the project.

Homework.

  • Get the code ready for npm.
Thanks @aircrewmentor for everything you are doing.

Node-Inspector, SMART & Testing.

Session 09-06-2015

Summary

  1. Node-Inspector
  2. SMART
  3. Testing Code

Session

1. Node-Inspector

Here we talked about the Node-Inspector that we covered on the previous session, I didn’t get that we had to press play on the browser’s debbuger to be able to send data to our code, so I was getting no response, but now everything is really clear.

I updated our talk from last week and I think I found a faster way of doing the debug, sure there is some price I am paying for doing it this way and I don’t know about yet. @aircrewmentor could tell me if it’s ok.

2. SMART

The SMART Criteria is a trend really useful for project management that can be summarized on the following:

  • Specific – target a specific area for improvement.
  • Measurable – quantify or at least suggest an indicator of progress.
  • Assignable – specify who will do it.
  • Realistic – state what results can realistically be achieved, given available resources.
  • Time-related – specify when the result(s) can be achieved.

we can get a better perception with the following image:

smart

3. Testing Code

We continued testing the code from the map application, I commented that I started following this guide and we went over the Atomus solution for a headless browser.

So we moved a little code on the before section and it became clear that I have a lot of work to do on mocha .

AirCrewMentor uses testem and remarked the importance of using TDD for everything.

So to debug the test we used again Node-Inspector with the following command.

$ mocha test/ -w --debug-brk

with the Node-Inspector opened like we explained before in last chapter we were able to debug the code as well.

I have found Karma to be a useful and trending tool, so we’ll see which one I end up using, for now I am going to start with the course TDD.

Remarks

Talking about the hard work and the talent, AirCrewMentor commented about a book called Talent is overrated that we can see a summary in this article that basically says that the talent doesn’t exist, only the hard work.

Homework.

  • TDD
Thanks for your time @aircrewmentor, wish you happy holidays.

Node-Inspector

Summary

  1. Show me the code
  2. Man in the middle
  3. Node-Inspector
  4. Pipe

Session

1. Show me the code

This is the what all the developers say when they want to know how good or what a person is working on.

This came up because I was explaining how this week I had done the nodeschool.io course.

2. Man in the middle

We talked a little bit about what is this concept, because all our connexions goes through different computers over the world, and with this technique hackers use to steal data.

To avoid this problems we use HTTPS protocol so the person who is in the middle can only see encrypted data.

3. Node-Inspector

To debug with NodeJS code, we have to use this tool Node-Inspector that when used, we can simply adding debugger; to our code, we’ll make it stop and open an inspector on our Browser.

This would be the process:

For this example we are going to use an exercise of learnyounode, where any data passed to the code, is returned in capital letters.

This is the code for the exercise:

var http = require('http')
var map = require('through2-map')
var server = http.createServer(function(req, res) {

  req.pipe(map(function(chunk) {
    return chunk.toString().toUpperCase()
  })).pipe(res)
})
server.listen(process.argv[2])

To start the server we are going to run the following command, passing as a second argument the port of the server.

$ node http_uppercaserer.js 3000

this will open the server and will let it waiting for us to work on it.

then we are going to pass to the sever some data with the following command:

$ curl --data AirCrew http://localhost:3000

this will automatically return AirCrew

But if we would like to test the code with breaks, we do the following:

Now we are going to add a debugger; in line 4

var http = require('http')
var map = require('through2-map')
var server = http.createServer(function(req, res) {
  debugger;
  req.pipe(map(function(chunk) {
    return chunk.toString().toUpperCase()
  })).pipe(res)
})
server.listen(process.argv[2])

now we can start the application with the following command:

$ node --debug-brk http_uppercaserer.js 3000
debugger listening on port 5858

in other tab of the terminal we use to run the code the command:

$ node-inspector 
Node Inspector v0.10.1
Visit http://127.0.0.1:8080/debug?ws=127.0.0.1:8080&port=5858 to start debugging.

then we open the browser on the link (remember to press command when click on iterm to go to the page.) and once is loaded, we press play to start debugging.

Then we can send as we did previously our command with the data so we’ll see the debugger on our browser stopping where we specified so we can start debugging like we would normally do on a normal javascript code executed on the browser instead of NodeJS.

A shorter and seems simple way of doing this would be the following:

Before continue we should do a killall node to be sure that there is no other activity running.

So, to simple launch the code with the debugger tool on the default browser, we would have to use the following:

$ node-debug http_uppercaserer.js 3000

This will launch the code and simply open the debugger view on the browser, once is loaded we would press play (F8) to start the debugger and then we’ll simple send the data like we did before

$ curl --data AirCrew http://localhost:3000

4. Pipe

So, Pipe it’s a really important tool that not a lot of people knows how to use, it is important because it reduces the size used.

On big files, the use of pipe is mandatory.

in a popular way, to know what pipe does, we can say that it takes a file or data, start a process with the first information that get from that file or data and as soon as get the first results, starts sending the results, it doesn’t wait for the file to be totally loaded to start or to get the results of the process done in all the data.

In a normal process without pipe, if we try to work on a file that is 1 GB large, we would have to upload the file completely to the server and then once is uploaded start processing the information.

we can get more information on the NodeJS API

Remarks:

  • using the key ESC in google chrome shows the console.
  • At iTerm using cmmd + click into a link, open the browser with the link.

Write your code like the maintainer is going to be a Serial Killer.

Homework.

  • TDD – James Shore
    • Unit testing small functions.
  • Work on the project (future portfolio)
Thanks for your time @aircrewmentor, this has been a good session. I feel like I am understanding a lot of things. Still a lot of work ahead.

Object Oriented Programming, Babel and FileSystem.

Summary

  1. Object Oriented Programing.
  2. Ways of creating an object (9?)
  3. Use of ES6 – Babel.
  4. Generate Data.

Session

We had a more relaxed (but no less intense) session, we started talking about history and where the the Object Oriented programming comes from.

1. Object Oriented Programing.

So in 1985 Steve Jobs founded Next and started talking about the wonderful features of this kind of object Oriented programming.

Here we can see him explaining it. https://www.youtube.com/watch?v=2kjtQnPqq2U

2. Ways of creating an object (9?)

This is a question that is often asked on interviews so I am going to list them all here (at least what I know or can find):

(to be continued)

3. Use of ES6 – Babel.

It is time to start learning to use JS6/ES6 (ExpressionStatement) so when it comes available I can have a good set of skills, for that we can use Babel.io, I am not sure how I am going to do it, since I really don’t know that well JS5.

I hope @AirCrewMentor can give me some more guidance or stablish the path to start using it.

ES6 Features
Babel.io

4. Generate Data.

Here comes the main subject of the session, how to start creating files from our data with node features.

So it is common convention to have a /bin folder where we can locate our binaries.

So we included a file with a sentence that is known as a shebang

#!/usr/bin/env node

With this we are passing the code that follows into node for its execution.

So we created a file without extension and added the code so it runs inside node:

#!/usr/bin/env node
var Papa = require('babyparse'),
    fs = require('fs');

function getAirports(callback) {
  var airportsCsv = (__dirname + "/../files/airports.dat");
  fs.readFile(airportsCsv, function (err, data) {
    if (err) {
      return callback(err);
    };
    Papa.parse(data.toString(), {
      complete: function(airports) {
        callback(null, airports);
      }
    });
  });
}

getAirports(function(err,airports){
  if (err) {
    throw err;
  }
  console.log(airports);
});

With this code we are using the FileSystem module from Node.

When we were going to execute the file on terminal typing just $ ./bin/generate_icao we find a problem, it wasn’t executing and it wasn’t automatically selected when tab was pressed.

This is because we didn’t have execution permissions, so we changed chmod +x filename.extension

Remarks:

I have to lint properly all the code. Using JSLint is a must.

Homework.

  • Get file Writing (I have to enable the automatically creation of files)
  • Use Async with limits for operations.
  • Read Callbacks
  • The art of Node
  • NodeSchool.io
Thanks @AirCrewMentor , I’ll hope next time I’ll be more agile, sometimes it is difficult.🙂

Map & Leaflet

Summary

  1. Finished refactoring airports
  2. Render airports
  3. Map.

Session

So we started the session with a couple of doubts I had from the book JS The good parts like recursion, so I am now starting chapter 5, really condensed. I want to understand every example, so I am testing and writing everyone of them.

Following I showed him how I have implemented the cluster option and the search functionality with Lunr, my mentor liked it but suggested a little CSS, like Cursor Pointer when passing over the results.

1. finished refactoring airports

We finished with the asynchronous function making the data from the airports file to load in an asynchronous way, so the map keeps going and you can interact with it really quick, and also if the data doesn’t load, the code doesn’t break, as it was happening before.

Now we are returning a data array with all the data from the airports inside the actual scope on the map. Before we were returning the markers.

As a result we have now a function that return us all the points from a data that are within the actual scope, the possibilities were we can use it are wider than returning only markers.

2. render airports

So we have divided every process, the map is generated in one file called map.js , the airports within the scope are generated from a file called locate_airports.js and finally to add markers to an array of points given, we are doing it in a separate file called render_airports.js.

So we were having an small problem because to generate the control, we were calling it every time this was being generated.

So the trick came when we established a variable within the scope of render_airports established to false and when we were calling the function to create layers, the first time it generates it, it was setting the value to true, so the next time, this function would exit without generating the control layers.

Here is the code:

var layersIncluded = false;

function createLayers (baseLayers) {
  if (layersIncluded) {return;};// here is the trick
  var airportsLayerGroup = L.layerGroup([markers]);
  var overAirports = {
    "Airports": airportsLayerGroup
  };
  layersIncluded = true;
  L.control.layers(baseLayers, overAirports).addTo(map);
}

it seems really obvious but I think it is a cool trick.

3. Map.

So this is how we call now the asynchronous functions and we generate the airports layer with its control.

function showAirports() {
  locateAirports(map, function(error, airports) {
    // this returns an array called airports,
    // specified on the callback function.
    if (error) { // this will show the error if it brakes.
      throw error;
    };
    //we need the previous function to have callback with the airports
    //so we can pass it to the next function to generate the markers.
    renderAirports(map, airports, baseLayers, function(error) {
      //here there is no callback returned,
      //we just want the layer to be updated.
      if (error) {
        throw error;
      };
    })
  });
}
showAirports();
map.on('moveend overlayadd', showAirports);

Remarks.

  • Always reduce each function to the simplest .
  • Async works really great. in fact AirCrewMentor use it in almost every project.
  • AirCrewMentor uses Lodash in every project.

Common Convention

  • Files are named with words and underscore to divide them.

Homework.

  • Merge the branch Locate Airports with the master
  • Async (Try examples and read the Documentation)
  • Cursor Pointer (CSS)
  • Script generate files for each airport.
  • Weight based on routes
  • Read Alex’s blog.
Thanks for your time @AirCrewMentor and your compliment.

So, I started with the bash course end of October and it’s been from December when we installed RescueTime that I have dedicated 288 h to Develop, research and learning. Probably more since the pdf application at work isn’t categorized as learning, because I don’t use it only for learning.

Our goal to become a developer is 1000 hours, and now I am somewhere between the 300 and the 400 hours and you gave me a big compliment, telling me that you were impressed by my progress and that with a project like the map, I am electable for a junior position.

So I am really pleased, because I know how difficult is to impress you, so now we have to get ready my portfolio and start looking for a junior position.

Again, thanks for all @AirCrewMentor

Lunr, Marker Cluster and Async

Summary

  1. lunr.js
  2. Marker Cluster
    Leaflet Improvement to display a lot of points
  3. Refactor code for async.

Session

I showed @AirCrewMentor the work done with the Map project using Leaflet and he liked how I managed to extract all the data and how it was working, despite the many problems there are, like global variables, no tests, no browserify and so on…

At least I got the maps working😛 .

2387097999-Screenshot 2015-05-06 14.24.22

To convert the CSV (.DAT) file into JSON I used the Papa Parse Software, that worked really good.

After that I enabled a search field that was able to center the map on the result.

3837629339-Screenshot 2015-05-06 14.25.05

1. lunr.js

Viewing the strict of the search field I enabled, he suggested to use Lunr as a search engine.

2. Market Cluster

AirCrewMentor suggested to use a Leaflet Improvement to display a lot of points that group them, really cool feature, looking forward to use it.

3. Refactor code for async.

Here is the code we are going to refactor:

var airports = Papa.parse("./files/airports.dat", {
  download: true,
  header: true,
  complete: function(results) {
    airports = results;
    locateAirports(airports);
  }
});

var markers = new L.FeatureGroup();

var locateAirports = function(airports) {
  console.log(map.getZoom());
  if (map.hasLayer(searchMarker)) {
    map.removeLayer(searchMarker);
  };
  if (map.getZoom() < 5) {
    markers.clearLayers();
    return;
  };
  markers.clearLayers();
  for (var i = 0, j = airports.data.length; i < j; ++i) {
    var airportLatitude = airports.data[i].Latitude,
    airportLongitude = airports.data[i].Longitude;
    if (map.getBounds().contains([+airportLatitude, +airportLongitude])) {
      var marker = L.marker([airportLatitude, airportLongitude])
      .bindPopup('
      <p class="airport_marker">' + airports.data[i]["Name"] + '</p>
      ');
      markers.addLayer(marker);
    }
  }
// if we want the airports to be always visible uncomment following line.
// map.addLayer(markers);
};
var airportsLayerGroup = L.layerGroup([markers]);
var overAirports = {
  "Airports": airportsLayerGroup
};

The big fail is that the code was not designed in an asynchronous manner and this can cause a lot of troubles and is not a best practice, I knew that when I was doing it, but my low experience made me focus on making it work rather than making it work the right way. I hope that getting more experience and better understanding of what asynchronous mean, I am going to be able to use it when properly required.

on asynchronous code we first treat the error case and later the good one.
Please @AirCrewMentor  check this. Thanks.

So AirCrewMentor  showed me a little trick that he uses to make the global variables to go away and at the same time use the asynchronous code.

(function (name, global, definition) {
  // On Node.js/CommonJS
if (typeof module !== 'undefined') {
  module.exports = definition(name, global);

  // On the browser
  } else {
  global[name] = self;
}
}("localeAirports", this, function () {
'use strict';

///////

function localeAirports(map, callback) {
}

///////

return localeAirports;
}));

So using it we started the refactor of the code.

About this refactor I have a couple of remarks I would like to point out.

AirCrewMentor was talking all the time about how I had mixed concepts and that was because the same function that was intended to give me the airports, was updating the map as well, so it seems a novice mistake, I should keep each function with an specific task, so we refactored it taking out the part where it was updating the map.

We got a little into the callback function that is needed on the asynchronous code.

  1. We wrote the function to convert the CSV file using Papa Parse in a way that if the airports variable already had the value asigned, it would use the callback to go back to the function that called it. Otherwise it will convert it, so this way we can assure the process only one time.
var airports;
///////
function getAirports(callback) {
  if (airports) {
    callback(null, airports);
    } else {
    Papa.parse("./files/airports.dat", {
      download: true,
      header: true,
      complete: function(results) {
        airports = results;
        callback(null, airports)
      }
    });
  }
}
  1. Converted the function so it was the locateAirports who was calling always the getAirports function, waiting for it to return the airports, otherwise exiting the function preventing the code to break.

  2. Then if the airports variable was returned it would proceed and it would finally give back the results, that we would use in other function to update the map. So we extracted the part of the function that was updating the map and let this one with a single purpose.

function locateAirports(map, callback) {
  getAirports(function(error, airports) {
    if (error) {
      callback(error);
      return;
    };
    var results = [];

    for (var i = 0, j = airports.data.length; i < j; ++i) {
      var airportLatitude = airports.data[i].Latitude,
      airportLongitude = airports.data[i].Longitude;
      if (map.getBounds().contains([+airportLatitude, +airportLongitude])) {
        var marker = L.marker([airportLatitude, airportLongitude])
        .bindPopup('
        <p class="airport_marker">' + airports.data[i]["Name"] + '</p>
        ');
        results.push(marker);
      }
    }
  callback(null, results);
  });
}

Remaks.

  • use of the command ls -hal where adding the h give use the unit of measure.

We get this result:

$ ls -hal
total 6408
drwxr-xr-x 5 username staff 170B May 3 00:55 .
drwxr-xr-x 17 username staff 578B May 4 15:34 ..
-rw-r--r-- 1 username staff 830K May 3 00:55 airports.dat
-rw-r--r-- 1 username staff 2.3M May 3 00:55 routes.dat

Instead of this:

$ ls -all
total 6408
drwxr-xr-x 5 username staff 170 May 3 00:55 .
drwxr-xr-x 17 username staff 578 May 4 15:34 ..
-rw-r--r-- 1 username staff 850419 May 3 00:55 airports.dat
-rw-r--r-- 1 username staff 2375638 May 3 00:55 routes.dat

Notice the 2.3M instead of 2375638. Much more cleaner and readable.

Homework.

  • Implement Cluster and Lunr.js
  • Read Javascript: The good parts
Thanks @AirCrewMentor . I am happy you liked the work.🙂