NoSQL: MongoDB vs DynamoDB for Big Data storage

Minor quote on basic comparison of MongoDB vs DynamoDB from Amazon:

Engines DynamoDB MongoDB
Data Model
  • Key-value with JSON support
  • Up to 400 kb record size
  • Limited data type support
  • JSON-like document
  • Up to 16 mb record size
Querying Key-value queries Query & analyze data in multiple ways — by single keys, ranges, faceted search, graph traversals, and geospatial queries through to complex aggregations

In both cases they perfectly suite for big data storage. Both offer cloud-based (AWS, and Mongo Atlas) and local storage solutions.

Positive feedback on DynamoDB and its pricing policy  (pricing calculator):

Amazon lets you buy operations per second capability rather than CPU hours or storage space. This removes a whole lot of complexity for developers who would otherwise need to tune the database configuration, monitor performance levels, ramp up hardware resources when needed. This provides users a fast and reliable storage space for their needs with costs that scale in direct proportion to the demand.

For now, DynamoDB looks more favorable. The fact that it’s an AWS-only engine doesn’t set any limits. The overall stack of AWS tools that exists in their infrastructure looks promising.

HubSpot: Gatekeepers and Gardeners

HubSpot tech blog published great article on job balancing and tech leads paradox of gatekeepers gardeners.

You might be a gatekeeper if:

  • your team regularly waits for you to review their PRs
  • your team waits to do the next thing assigned to them instead of taking initiative to find projects for themselves
  • you hesitate to go on vacation because you’re concerned your team will struggle in your absence

On the opposite.

A gardener might:

  • forego reviewing work, or let other members of the team take on the responsibility
  • let the team handle their own task management, trusting they understand the needs of the customer, business, and team
  • encourage members to build relationships on and off the team
  • let the team experience failure, trusting in their accountability to fix their problems and learn from their mistakes
  • have their team take on grungy work along with the “fun” work, because they understand the value of it

Great overview of these two roles people occasionally take once becoming managers/tech leads.

Amazon AWS: Giant dilemma in IT

Microsoft, Google, Apple. Now Amazon. Whenever IT company becomes a leader in its niche, people start grumbling about its “evil” intents or monopolizing the market. Well, it’s natural for a market (i.e. capitalist) model. Monopoly kills competition, but to become a monopolist you have to really work hard to reach it.

Same happened with Amazon recently.

Amazon is a retailer, price sensitivity is in their DNA. In the past, when new companies like Digital Ocean have gotten large enough to show up on their radar they’ve reacted with deep price cuts.

….

Adding this all up, Amazon is terrifying to compete with.

 

 

Main Software Engineering cornerstone: Quality

Recently Paul M. Jones published great article on the Software Engineering major conflict between Business and Software worlds: Product quality that perfectly matches Robert L.Glass book I’ve been recently reading on “Facts and Fallacies of Software Engineering“. If you still didn’t read it – strongly recommended!

Just few excerpts on the article:

They [engineers] have a reputation to maintain. Low quality for their kind of work is bad for reputation among their peers. (Note that their peers are not necessarily their employers.)

They understand they may be working on the same project later; higher quality means easier work later, although at the expense of (harder? more?) work now.

And on the other side (dark side?..):

The reputation of the payer is not dependent on how the work is done, only that the work is done, and in a way that can be presented to customers. Note that the customers are mostly not the programmer’s peers.

Some useful links for JS coding

Few nice things to use in close future:

Hoodie

A fast, simple and self-hosted backend as a service for your (web) apps, Open Source and free. No need to write server-side code or database schemas. Makes building offline-capable software a breeze.

GREENKEEPER.IO

Get safety & consistency with real-time monitoring and automatic upates for npm dependencies.

Reduce CSS bundle size by cutting class names

This is pure awesomeness! Gajus Kuizinas went beyond asset minifiers which resulted from 140KB to 53KB CSS size decrease.

There is one thing a minifier cannot do – change the selector names. This is because a CSS minifier does not control the HTML output. Meanwhile, CSS names can get long.

All the details are in the post worth reading, if still think that webpack can only merge JS files in a bundle.

JSData as ember-data alternative

Weekends are normally for RnD time, so this weekend’s topic – standardization of front/backend communication. Considering backend as API-only and fully offloading the rendering part to JavaScript frameworks, EmberJS had a really nice concept of ember-data that acted in between rendering components and backend objects received via API. Another alternative can be considered as JSData:

SData is a framework-agnostic, datastore-agnostic ORM (Object-Relational Mapper) for Node.js and the Browser.

Adapters allow JSData to connect to various data sources such as Firebase, MySql, RethinkDB, MongoDB, localStorage, Redis, a REST API, etc. With JSData you can re-use your data modeling code between environments, keep your data layer intact when transitioning between app frameworks, and work with a unified data API on the server and the client. JSData employs conventions for rapid development but allows for endless customization in order to meet your particular needs.

Considering the use of JSData with JSON-API standards, there’re couple of adapters that transpile JSData to JSON-API.

CakePHP with NightwatchJS on Travis CI

After the release of Headless Chrome browser (v59+) most of the automation tools slowly started reducing the usage of Selenium, by replacing it with the combination of chromedriver + chrome/chromium pre-installed browser (like Travis CI does on their images).

As a replacement, I’ve tried playing around with the combination of Karma/Chai to test JavaScript functionality of certain projects we have in Qobo, which went well, but it only introduced the unit test approach towards the code base. What I wanted was user automation, emulating browsers Page Object Model (POM). The choice fell on NightwatchJS library that works with headless Chrome and needs only minor tweaking.

Setting up NightwatchJS

I’ve used a combination of NightwatchJS with Mocha test framework, but Nightwatch itself has a good support of assertion methods to do some basic checks on the browser requests, but first things first.

We start with package.json for Yarn/NPM.

{
  "devDependencies": {
    "chromedriver": "^2.31.0",
    "mocha": "^3.5.0",
    "nightwatch": "^0.9.16"
  },
  "scripts": {
    "test": "./node_modules/nightwatch/bin/nightwatch"
  }
}

yarn install and proceeding with nightwatch configurations:

const chromedriver = require('chromedriver');

module.exports = {
  before: function(done) {
    chromedriver.start();

    done();
  },

  after: function(done) {
    chromedriver.stop();

    done();
  }
};

After we explained the library how to handle chromedriver, we set the configurations for nightwatch.json:

{
  "src_folders": ["tests/Frontend"], 
  "output_folder": "build/coverage/",
  "custom_commands_path": "",
  "custom_assertions_path": "",
  "page_objects_path": "",
  "globals_path": "globals.js",
  "selenium": {
    "start_process": false
  },
  "test_runner": {
    "type": "mocha",
    "options": {
        "ui": "bdd",
        "reporter": "list"
    }
  },
  "test_settings": {
    "default": {
      "selenium_port": 9515,
      "selenium_host": "localhost",
      "default_path_prefix" : "",

      "desiredCapabilities": {
        "browserName": "chrome",
        "chromeOptions" : {
          "args" : ["--no-sandbox", "--headless", "--disable-gpu"]
        },
        "acceptSslCerts": true
      }
    }
  }
}

NightwatchJS still uses selenium_` naming options in the configurations (for backward compatibility reasons), so don’t get scared by the naming.

And the final touch, testing login action:

describe('Testing login UsersController::login() method', () => {
    var loginUrl = 'http://localhost:8000/login';

    before(function(browser, done) {
      done();
    });

    after(function(browser, done) {
      browser.end(function() {
        done();
      });
    });

    afterEach(function(browser, done) {
      done();
    });

    beforeEach(function(browser, done) {
      done();
    });

    it('gets [login] page', (browser) => {

        browser
            .url(loginUrl)
            .waitForElementVisible('.login-box').present;
    });

    it('trying to [login]', (browser) => {
        browser
            .url(loginUrl)
            .waitForElementVisible('.login-box', 2000)
            .assert.elementPresent('input#username')
            .setValue('input#username', 'username')
            .assert.elementPresent('input#password')
            .setValue('input#password', 'password')
            .submitForm('form')
            .pause(2000)
            .assert.elementPresent('nav');
    });
});

Running yarn test you should get something like this:

running nightwatchjs with yarn

NightwatchJS on Travis CI

Last but not the least – configuring Travis CI to run NightwatchJS on a PHP distro image. The config is partially pasted from project-template-cakephp , (pull request #340 with NightWatchJS).

sudo: true
dist: trusty

language: php

php:
    - 5.6
    - 7.0
    - 7.1
    - nightly

#enabling NodeJS
node_js:
  - "7"

#setting up stable Chrome, which has headless support 
#out of the box
addons:
  chrome: stable

#caching Yarn and its node_modules
cache:
  yarn: true
  directories:
    - node_modules

# installing JS packages and starting up the server
before_script:
    - yarn install
    - ./bin/phpserv >/dev/null 2>&1 &
    - sleep 5

# once you're done with PHPUnit/PHPCS,
# it's time to check the UI with `yarn test`
script:
  - ./vendor/bin/phpunit --group example
  - ./vendor/bin/phpunit --exclude-group example
  - ./vendor/bin/phpcs
  - yarn test

 

ReactJS: BSD + Patents licensing model

The devil is in details, so people say. Few months ago, while working on prototype of cakephp-calendar, we had an interesting debate over front-end stack for Calendar component. At the same time, in parallel, WordPress community was buzzing about their text editor – Gutenberg whether to use ReactJS vs VueJS libraries. One of the reasons of choosing VueJS, was licensing agreement.

The license granted hereunder will terminate, automatically and without notice,
if you (or any of your subsidiaries, corporate affiliates or agents) initiate directly or indirectly, or take a direct financial interest in, any Patent Assertion…(c)

Investing man-hours into VueJS research and prototyping was right, especially after Raul Kripalani published an interesting article on the license review of ReactJS.

Updated: One more review of FB React JS licensing from WordPress Tavern.

VueJS watchers for MomentJS

It took me some time to figure out what’s been going wrong with the watchers working on datepicker integration for CakePHP-Calendar plugin.

Problem domain:

On loading modal window for creating calendar event, one of the dropdowns pass default start/end time for the event interval (start/end dates). This config object is passed to as a property into interval component that contains two child datepicker components.

Whenever the config property changes, it adjusts startMoment and endMoment objects and passes them into datepickers. The change is monitored via watch functionality of VueJS.

Example:

Vue.component('interval-holder', {
    template: `<div>
        <datepicker :date-moment="startMoment"></datepicker>
        <datepicker :date-moment="endMoment"></datepicker>
    </div>`,
    props: ['config', 'initialMoment'],
    data: function() {
        return {
            startMoment: null,
            endMoment: null
        };
    },
    watch: {
        config: function() {
            this.startMoment = this.initialMoment;
            this.startMoment.set({'hour': this.config.start.hour, 'minute': this.config.start.min});
            
            this.endMoment = this.initialMoment; 
            this.endMoment.set({'hour': this.config.end.hour, 'minute': this.config.end.min});
        }
    }
});

Vue.component('datepicker', {
    // value var is useless here, as we use daterangepicker from jQuery 🙁
    template: `<div>
        <input type="text" :value="value"/>
    </div>`,
    props: ['dateMoment'],
    mounted: function() {
        var self = this;
        //load jquery bootstrap-datepicker instance
        this.instance = $(this.$el).find('input').daterangepicker().data('daterangepicker');
        
        $(this.$el).find('input').on('apply.daterangepicker', function(ev, picker) {
            self.momentObject = picker.startDate;
            self.value = picker.startDate.format('YYYY-MM-DD HH:mm');
        });
    },
    data: function() {
        return {
            instance:null,
            value: null,
            momentObject: null,
        };
    },
    watch: {
        dateMoment: function() {
            this.momentObject = this.dateMoment;
            this.value = this.momentObject.format('YYYY-MM-DD HH:mm');
        },
    }
});

If we set hours and minutes the following way as in the example above, dateMoment watcher won’t catch the change. According to the docs, it’s adviced to use deep param:

To also detect nested value changes inside Objects, you need to pass in deep: true in the options argument. Note that you don’t need to do so to listen for Array mutations.

watch: {
    dateMoment: {
        handler: function(val, old) {
            console.log('dateMoment changed');
        },
        deep: true
    }
}

Still no luck. Apparently, MomentJS object “too deep”. The only way to get it working ended up like this, change the parent component moment objects assignment, keeping the child-component watch function unchanged.

watch: {
    config: function() {
        var newStart = moment(this.startMoment);
        newStart.set({'hour': .., 'minute': ..});
        this.startMoment = newStart;
        
        //create a new instance of newEnd and assigning it to this.endMoment
        // get the dateMoment observe the change.
        // Ugly? Yes. Working? Yes..
    }
}

If anyone has a better (more elegant) way to fix this issue – welcome in the comments section! 🙂