Wednesday, December 05, 2018

docker: Error response from daemon: OCI runtime create failed:

Problem:

$ docker run -i -t a4a9a769aa01 /bin/bash
docker: Error response from daemon: OCI runtime create failed: container_linux.go:348: starting container process caused "exec: \"/bin/bash\": stat /bin/bash: no such file or directory": unknown.

Resolution:

$ docker run -i -t a4a9a769aa01 sh

Docker Compose Version 2 and 3

Docker Compose file structure for version 3 is the same as version 2. Some of the items in version 2 are removed in version 3. For more details: Docker Compose File Upgrade

Docker Networking Basics

$ docker version
Client: Docker Engine - Community
 Version:           18.09.0
 API version:       1.39
 Go version:        go1.10.4
 Git commit:        4d60db4
 Built:             Wed Nov  7 00:47:43 2018
 OS/Arch:           darwin/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          18.09.0
  API version:      1.39 (minimum version 1.12)
  Go version:       go1.10.4
  Git commit:       4d60db4
  Built:            Wed Nov  7 00:55:00 2018
  OS/Arch:          linux/amd64
  Experimental:     true
  
$ docker-compose version
docker-compose version 1.23.2, build 1110ad01
docker-py version: 3.6.0
CPython version: 3.6.6
OpenSSL version: OpenSSL 1.1.0h  27 Mar 2018
$ docker network

Usage: docker network COMMAND

Manage networks

Commands:
  connect     Connect a container to a network
  create      Create a network
  disconnect  Disconnect a container from a network
  inspect     Display detailed information on one or more networks
  ls          List networks
  prune       Remove all unused networks
  rm          Remove one or more networks

Run 'docker network COMMAND --help' for more information on a command.
$ docker network ls
NETWORK ID          NAME                   DRIVER              SCOPE
2ecc545ad2b8        alpy_default           bridge              local
d8671393ed7e        bridge                 bridge              local
b3c19c5fabc5        host                   host                local
861cdc4e1aaa        kafk_default           bridge              local
43c34d2fd46b        kafka-docker_default   bridge              local
4436df0f00b3        none                   null                local
7b3893e43bb7        rmoff_default          bridge              local


$ docker network inspect kafk_default
[
    {
        "Name": "kafk_default",
        "Id": "861cdc4e1aaaed3e874a93e0153fdf2eeaabb9a81c84fe246c0db1e83408abbb",
        "Created": "2018-12-04T23:38:29.1987718Z",
        "Scope": "local",
        "Driver": "bridge",
        "EnableIPv6": false,
        "IPAM": {
            "Driver": "default",
            "Options": null,
            "Config": [
                {
                    "Subnet": "172.20.0.0/16",
                    "Gateway": "172.20.0.1"
                }
            ]
        },
        "Internal": false,
        "Attachable": false,
        "Ingress": false,
        "ConfigFrom": {
            "Network": ""
        },
        "ConfigOnly": false,
        "Containers": {},
        "Options": {},
        "Labels": {}
    }
]

$ docker info
Containers: 20
 Running: 4
 Paused: 0
 Stopped: 16
Images: 48
Server Version: 18.09.0
Storage Driver: overlay2
 Backing Filesystem: extfs
 Supports d_type: true
 Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: bridge host ipvlan macvlan null overlay
 Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 468a545b9edcd5932818eb9de8e72413e616e86e
runc version: 69663f0bd4b60df09991c08812a60108003fa340
init version: fec3683
Security Options:
 seccomp
  Profile: default
Kernel Version: 4.9.125-linuxkit
Operating System: Docker for Mac
OSType: linux
Architecture: x86_64
CPUs: 2
Total Memory: 1.952GiB
Name: linuxkit-025000000001
ID: SPSV:BXUV:N7OG:TBYX:CGRI:RDXB:F7RW:JTRP:ZUBN:UAM2:7WD4:Z5GQ
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): true
 File Descriptors: 51
 Goroutines: 82
 System Time: 2018-12-05T17:17:10.0243027Z
 EventsListeners: 3
HTTP Proxy: gateway.docker.internal:3128
HTTPS Proxy: gateway.docker.internal:3129
Labels:
Experimental: true
Insecure Registries:
Live Restore Enabled: false
Product License: Community Engine

Docker on Mac. Version is most recent than Docker Networking course by Nigel Poulton.

Tuesday, December 04, 2018

How to find Docker host IP

On Mac:

$ docker run -it busybox /bin/sh

Inside the container:

/ # pwd
/
/ # ping docker.for.mac.localhost
PING docker.for.mac.localhost (192.168.65.2): 56 data bytes
64 bytes from 192.168.65.2: seq=0 ttl=37 time=6.215 ms
64 bytes from 192.168.65.2: seq=1 ttl=37 time=0.698 ms
64 bytes from 192.168.65.2: seq=2 ttl=37 time=0.423 ms
64 bytes from 192.168.65.2: seq=3 ttl=37 time=0.324 ms
64 bytes from 192.168.65.2: seq=4 ttl=37 time=0.647 ms
^C
--- docker.for.mac.localhost ping statistics ---
5 packets transmitted, 5 packets received, 0% packet loss
round-trip min/avg/max = 0.324/1.661/6.215 ms

On Mac:
$ export HOST_IP=192.168.65.2
$ echo $HOST_IP
192.168.65.2
$ docker run --add-host outside:$HOST_IP --name busybox -it busybox /bin/sh
/ # cat /etc/hosts
127.0.0.1 localhost
::1 localhost ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
192.168.65.2 outside
172.17.0.4 b0d28743cbcb

Reference




Friday, November 30, 2018

ERROR: could not find idn library!

CentOS issue resolution:

wget -q https://ftp.gnu.org/gnu/libidn/libidn-1.35.tar.gz tar xfz libidn-1.35.tar.gz

tar xfz libidn-1.35.tar.gz

cd libidn-1.35/
./configure
make
make install

 References 
  libidn

make: g++: Command not found

yum groupinstall 'Development Tools'

Friday, November 23, 2018

Active Class Does not Highlight in Bootstrap

For some reason, the Bootstrap active class was not highlighting the tab in Rails project. The workaround:

Use style='background-color: #5e5e5e' or use application helper:

style='<% active_background(account_path) %>'

Friday, November 16, 2018

Simple Dockerfile and docker-compose.yml for Rails 5 App

Dockerfile contents:

FROM ruby:2.5

RUN apt-get update -qq && apt-get install -y build-essential libpq-dev nodejs

RUN mkdir /budget
WORKDIR /budget

COPY Gemfile /budget/Gemfile
COPY Gemfile.lock /budget/Gemfile.lock

RUN bundle install
COPY . /budget

LABEL maintainer="Bala Paranj "

CMD puma -C config/puma.rb

docker-compose.yml:

version: '3'
services:
  web:
    build: .
    command: bundle exec rails s -p 3000 -b '0.0.0.0'
    ports:
      - "3000:3000"

Run:

docker-compose build
docker-compose up

Simple CircleCI Configuration File to Build Rails 5 Project

I am working on the landing page for Budget Scape. Here is the simplest circleci/config.yml:

version: 2
jobs:
  build:
    working_directory: ~/budget
    docker:
      - image: circleci/ruby:2.4.1
        environment:
          RAILS_ENV: test
    steps:
      - checkout

      - run: sudo apt-get update && sudo apt-get install nodejs
      # Bundle install dependencies
      - run: bundle install

      # Run the tests
      - run: bundle exec rake

I am using sqlite as the production database.  The home page is a lead capture page with bullet list of benefits for a user.

Wednesday, November 07, 2018

Developing REPL from Scratch in Ruby

Read Evaluate Print Loop

One liner:

loop { p eval gets }

Readable:

loop do
  print "$ "
input = gets.chomp!
result = eval(input)
puts "=> #{result}"
end


Persist Local Variable

myirb_binding = binding()
loop do
  print "$ "
input = gets.chomp!
result = myirb_binding.eval(input)
puts "=> #{result}"
end

Code written by Chris Wanstrath:

loop do
  print ENV['REPL_PROMPT'] || "#{ARGV[0]}>> "

  begin
    line = $stdin.gets.chomp
  rescue NoMethodError, Interrupt
    exit
  end

  if from_stdin
    run = "echo \"%s\" | #{command}" % [ line, nil ]
  else
    run = "#{command} %s" % [ line, nil ]
  end
  puts "$ #{run}" if debug
  system run
  warn "Use Ctrl-D (i.e. EOF) to exit" if line =~ /^(exit|quit)$/
end

Difference Between IRB and Code in Ruby File

One of the differences is that any method defined at the top level in a Ruby file becomes a private method in Object.

def greet
  p 'hi'
end

o = Object.new
o.greet

NoMethodError: private method ‘greet’ called for #. You can verify this:

 def greet
   p 'hi'
 end 

 p self.private_methods.grep(/greet/)

But in IRB it becomes a public method.

$ irb
2.3.3 :001 > def greet
2.3.3 :002?>   p 'hi'
2.3.3 :003?>   end
 => :greet
2.3.3 :004 > o = Object.new
 => # 
2.3.3 :005 > o.greet
"hi"
 => "hi"
 
Hey, you can also use self to call the greet method like this:

self.greet

or just

greet


Tuesday, November 06, 2018

Moving a Repo Quickly from Gitlab to Github

If you don't mind losing the git log in the new repo, you can move a repo easily in three steps:

1. Remove the existing git files in the repo:

rm -rf .git

2. Add a new origin:

git remote add new-origin git@github.com:your-user:your-project.git

3. Push it to the new repo

git push --all new-origin

Managing SSH Keys in Ubuntu

sudo apt-get install keychain

In ~/.bashrc, add:

keychain id_rsa
. ~/.keychain/`uname -n`-sh

In the terminal:

source ~/.bashrc

 * keychain 2.8.2 ~ http://www.funtoo.org
 * Found existing ssh-agent: 5059
 * Known ssh key: /home/bparanj/.ssh/id_rsa

I had already generated the ssh key using ssh-keygen and ran:

$ eval "$(ssh-agent -s)"
Agent pid 5059 
 



Monday, November 05, 2018

kazam on Ubuntu to record screencast

It works. But the size of the files that it creates is huge. It quickly filled up 100 GB hard disk space allocated to the Ubuntu 18.04 running on VirtualBox quickly. Started getting out of disk space errors that I was scratching my head to fix for a few days.

How to find available disk space in Ubuntu

df -h --total

This command will show you:

Filesystem      Size  Used Avail Use% Mounted on
total           105G  8.6G   92G   9% - 
 
 

Sunday, October 28, 2018

Ruby Closure - Outline for Presentation #2


Part 1

Required Concepts 

1. yield
2. yield with value
3. block variable
4. Migration file uses both Object Oriented and Functional style of programming.

In a Rails project the migration file looks like this:

class CreateArticles < ActiveRecord::Migration

  def change
    create_table :articles do |t|
       t.string :title
       t.text :body

       t.timestamps
     end
  end
end


The final solution for the presentation:

module ActiveRecord
  class Migration
    def change
      puts 'change'
    end

    def create_table(name)
      table = Table.new(name)
      yield table
    end
  end
end

class Table

  def initialize(name)
    @name = name
  end

  def string(column_name)
    puts "String column #{column_name} for table #{@name}"
  end
end

class CreateArticles < ActiveRecord::Migration
  def change
    create_table :articles do |t|
      t.string :title
    end
  end
end

c = CreateArticles.new.change

Implementing the text and timestamps are left as exercises for the attendees.

Part 2

Required Concepts

1. Changing the value of self.
2. Executing code in a different context
3. Using instance_eval to change self
4. yield vs instance_eval
5. Taking block in the argument of a method
6. Using ampersand in the argument of a method
7. The significance of &block in the method argument

Routes file looks like this:

Rails.application.routes.draw do
  get '/home' => 'welcome#home'
end


Step 1


module Rails
  def draw
    puts 'routes file' 
  end
end

class Tester
  include Rails
end

t = Tester.new.draw

Step 2

module Rails
  def draw
    yield
  end
end

class Tester
  include Rails
end

Tester.new.draw do
  get '/home' => 'welcome#home'
end

Step 3

module Rails
  class Router
    def get(hash)
      p 'Method to handle HTTP get request'
    end  
  end

  def draw
    router = Router.new
    yield router
  end
end

class Tester
  include Rails
end

Tester.new.draw do |router|
  router.get '/home' => 'welcome#home'
end

The older versions of Rails used the block variable. How to get rid of it and simplify the DSL?

Step 4

Let's move the draw method into the Router class.

module Rails
  class Router
    def get(hash)
      p 'Method to handle HTTP get request'
    end  
    
    def draw
      yield self
    end
  end
end

Rails::Router.new.draw do |router|
  router.get '/home' => 'welcome#home'
end

Step 5

module Rails
  class Router
    def get(hash)
      p 'Method to handle HTTP get request'
    end  
    
    def draw(&block)
      instance_eval(&block)
    end
  end
end

Rails::Router.new.draw do 
  get '/home' => 'welcome#home'
end

We have replaced the yield with instance_eval that evaluates the given block in the context of the Router instance. It switches the value of self from main Object to Router object.

Part 3


Symbol to proc trick, for instance : ['ruby', 'powerful'].each(&:upcase) uses the ampersand colon trick. 

Concepts Required

1. Coercion
2. Using send for sending messages to an object. Why use send?
3. Significance of ampersand as the prefix to an object in a method argument
4. How does to_proc get invoked?
5. *args in the method argument.
6. Providing default value for block variable.
7. Using send to message an object.

1.

result = ['hi', 'how'].map {|x| x.upcase }
p result

2.

#  {|x| x.upcase }
result = ['hi', 'how'].map(&:upcase)
p result

3.

Goal :

map {|x| x.upcase } ----> map(&:upcase)

4.

The & in the argument converts the object to a Proc object by calling to_proc on it. The object is symbol. We need to implement to_proc in Symbol object.

class Symbol
  def to_proc
    Proc.new {|o| o.upcase }
  end
end

result = ['hi', 'how'].map(&:upcase)
p result

5. 
  def to_proc
    Proc.new {|o| o.send(self) }
  end

6. Multiple block variables

output = [1,2,3,4].inject(0) {|result, element| result + element }
p output

===

output = [1,2,3,4].inject(&:+)
p output

will break the existing to_proc implementation.

Fix:

  def to_proc
    Proc.new {|o, args| o.send(self, args) }
  end

But this will break:

result = ['hi', 'how'].map(&:upcase)
p result

Fix:

  def to_proc
    Proc.new {|o, args=nil| o.send(self, *args) }
  end

Now both one block variable and two block variables can be used.

Closure

A closure is in it's simplest form a block of code that can be passed around as a value, and that can reference variables in the scope it was created in even after exiting from that scope.

The variables are allocated on the heap. Because normally local variables live in the stack and are gone after a method returns.

References

How to Implement Closures
Closure in Crystal

Wednesday, October 10, 2018

Find all the Replace by Chaining ack output to sed

From the command line, I was able to replace Bootstrap 3 classes with Bootstrap 4 class using
ack, sed and xargs. 
 
ack "search text" -l --print0 | xargs -0 -n 1 sed -i "s/search text/replacement text/"
 
Reference:
 
Sed recursively with the help of ack and xargs
 

Sunday, October 07, 2018

Top 3 Lessons from Successful Open Source Projects

  1. Every good work of software starts by scratching a developer's personal itch.
  2. Good programmers know what to write. Great ones know what to rewrite (and reuse).
  3. Smart data structures and dumb code works a lot better than the other way around.
Linux was not written from scratch. Linus Torvalds took the Minix as the starting point. It is easier to start from a good partial solution. Minix provided the scaffold and eventually the Minix code was either rewritten or deleted. Read more at Cathedral Bazaar

Sunday, September 30, 2018

Organized Topics for Coding Skills

Big O Notation

Construction of Loops
Establishing Initial Conditions for Loops
Finding the Iterative Construct
Termination of Loops
Iteration Trace Table
Iterative Program Structure
While Loop Structure
For Loop Structure
For Loop Examples
Comparison: Loops vs Recursive Functions

Divide-and-Conquer
Recursion : Algorithmically, Semantically
How to Code Recursively
Stack Overflow Example
Compute a to the power b
Induction and Recursion

Summation Algorithm
Add a sequence of numbers
Recursion
Iteration
Recursive Sum

Multiplication
Iterative Solution
Recursive Solution

Division without divide operator
Trace : Desk Check a solution
Non-Linear Recursion

Factorial
Recursive Solution
Optimized Solution
Factorial Call Stack
Factorial Algorithm
Factorial Implementation

Compute a to the power b
Recursion Solution
Iterative Solution

Compute a % b
Iterative Solution
Efficient Solution

Fibonacci : Multiple Bases Cases - Recursion
Efficient Fibonacci
Fibonacci Runtime
Fibonacci Number Efficiency
Fibonacci Iterative Solution

Dynamic Programming

Square Root Desk Check Table
Binary Recursion
Even Number
Square Root
Linear Recursion

Exponentiation Function
Factorial C
Array C
Binary Search C
Reverse String C
Sum the Digits

Find Repeating Elements
Identical
Unique Characters
Unique Characters with Constraints
Duplicate Elements in Array
Recognize Non-Linear Recursion
Add Adjacent Elements
Runtimes + , *
Powers of 2 from 1 through n
Remove White Spaces
Remove Duplicates in a String
String Segmentation
Palindrome
Palindrome Substrings Diagrams
Palindrome

Tuesday, September 25, 2018

How to override merge conflict

To use the latest git version of the file. We can replace file that has conflict with the latest from repo by running these commands:

git reset -- path/to/file.ext


git checkout path/to/file.ext

Sunday, September 23, 2018

Unable to download data from https://rubygems.org/ - SSL_connect returned=1


gem install rails -v 4.1


ruby -ropen-uri -e 'eval open("https://git.io/vQhWq").read'


-e:1:in `eval': uninitialized constant OpenSSL::OPENSSL_LIBRARY_VERSION (NameError)

ruby -v -e 'require "openssl"; p OpenSSL'

ruby 2.1.2p95 (2014-05-08 revision 45877) [x86_64-darwin13.0]
OpenSSL


brew rm openssl
brew cleanup openssl
brew install openssl

Thursday, September 20, 2018

Monday, September 17, 2018

Copy / Paste terminal utility for Ubuntu

1. sudo apt-get update
2. sudo apt-get install xclip

Add an alias to the .bashrc file:

alias pbcopy='xclip -sel clip'

I am using the same command that is used on a Mac as the alias. I also have open alias to open a directory from a terminal:

alias open='xdg-open'

Wednesday, July 25, 2018

no implicit conversion of nil into String

This can happen when you have not defined the required configuration in secrets.yml. For instance, you are using http basic auth and the credentials are not defined in the config file.

Wednesday, July 18, 2018

simplehttpserver npm



npm install simplehttpserver -g
simplehttpserver .
simplehttpserver . -p 8081

Tuesday, July 17, 2018

Converting .avi file to .mp4

Install ffmpeg:

On Mac 10.10.5:


brew install ffmpeg



Thursday, July 12, 2018

Sprockets::FileNotFound: couldn't find file 'fingerprintjs2' with type 'application/javascript'

jquery_ujs Uncaught TypeError: $.ajaxPrefilter is not a function

This happened when I copied the started template from Bootstrap 4.1 version. It gives you the slim version of jQuery by default. Use the regular jQuery. https://code.jquery.com/jquery-3.3.1.min.js

In style.scss, include
@import bootstrap

Copy the bootstrap.js to vendor/assets/javascripts directory. Copy bootstrap.css to vendor/assets/css directory.

Sunday, July 08, 2018

NeverBounce API



s3 images





Overcoming Mental Hurdles to Software Development

Facing Too Many Unknowns 


Facing many unknowns and working on a new API can be overwhelming. Even if you have the physical and mental energy to work, you can get stuck. I faced this challenge in a recent project. I decided to tackle this by doing an easy task. I copied the basic Bootstrap 4.1 table with hard-coded data and rendered the view in the web app. I was able to see something real, think through vague requirements and asked the stake holder the right questions. If you have a client who only has a vague idea of what they need, it becomes a challenge to gain understanding with minimum amount of code.

Gaining Momentum


I was able to gain momentum by tackling simple tasks related to that feature. Refined the requirements and the user's flow over a few days to come up with more followup questions to finalize the requirements. Once the requirements became clear, I was not coding based on some phantom requirements and designing the complicated database structure that was not even required. The time spent on solving problems that did not exist was very energy draining. I had already spent time on analyzing the flow that could vary due to three variables. This made the solution very complex due to the amount of variations. The code was becoming a big mess.

Focusing on the Core


I drew simple diagrams with boxes and arrows to show the user's flow for different use cases. Discussion with the stakeholder had a structure because we focused only on one use case at a time.  There was no discussion of technology during this conversation. Only golden path scenario was discussed to clarify the requirements. We can add the alternate scenarios later. We can ask questions based on the existing diagrams, reason about what should happen and draw the alternate scenarios separately.

Gaining Business Context


I made the mistake of copying the similar features of other existing products. I realized it was a mistake later because the business context was different. During the conversation based on simple diagrams I was able to understand the business context. I had narrowed the scope of the problem to be solved.  I also pushed back on some requirements and asked the stakeholder to justify why it should be included in version 1. Some of the things might happen once every 3 months does not justify automation and was removed from the stated requirements.

Building a Prototype First


I did not build a prototype because it was a lot easy to use Bootstrap 4.1 with Rails to build the screens with hard coded data. I saved the time to learn a prototyping tool. I was able to focus on learning new AWS APIs that was required to build the product.

Being Agile and Lean


The client was remote and was able to chat via What's app and email few times a week. The requirements were in the emails and were not well thought out to write the test first. Clarify First, Test Later. I did not automate the infrastructure management because at this point I am not sure if I can get 99 clients who will find this product useful without any customization. It has been custom built for this particular client. Reading the lean startup articles, I was confident that I was targeting the right segment in the adoption curve.

Lessons Learned


  • Do not write production level code before understanding the business. 
  • Draw simple diagrams with technical details to gain business context first.
  • Narrow down the scope of the problem before starting to code.
  • It's ok to start the first few versions of the screen with existing products from similar domains.
  • Remember, these screens need to refined once the business context knowledge is acquired.
  • If the complexity of the solution is getting out of control, go back and see if you can push back on the requirements.


Friday, July 06, 2018

Web Apps and MVC

These diagrams explain the flow of the web app for different use cases. These diagrams can easily be understood by stakeholders with no technical background. The conversation based on these diagrams are useful to understand the requirements and clarify any vague requirements.

This flow is also useful when developing a MVC based Web app. Because these use cases can span across different controllers and each controller can be shared by different use cases. It provides a structure and minimizes the rework required in later stages after the app is deployed.