Amazon Route 53 Basics

Route 53

1. Register domain name
2. Create DNS Records

Configuring Route 53
Create hosted zone in Route 53 (Ex: clickplan.net)
Create resource records

3. Update domain registrar with name servers

Set low TTL with registrar

https://oliverhelm.me/sys-admin/updating-aws-dns-records-from-cli

You have installed and configured the AWS cli tools & the credentials you are using have the permissions necessary to make updates.

http://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html

aws route53 create-hosted-zone --name example.com --caller-reference 2014-04-01-18:47 --hosted-zone-config Comment="command-line version"

aws configure

aws help

Install AWS CLI

pip install awscli

Configure AWS CLI

aws --version

aws route53 create-hosted-zone --name www.clickplan.net --caller-reference sell-digital-products

aws route53 change-resource-record-sets --hosted-zone-id "/hostedzone/COPY12XYHERE" --change-batch file://changebatch.json

{

 "Comment" : "Create 

}

{
  "Comment": "Create an alias",
  "Changes": [
    {
      "Action": "CREATE",
      "ResourceRecordSet": {
        "Name": "www.clickplan.net",
        "Type": "A",
        "AliasTarget": {
          "HostedZoneId": "hosted zone ID for your CloudFront distribution, Amazon S3 bucket, Elastic Load Balancing load balancer, or Amazon Route 53 hosted zone. Something like Z#BJ6K6RIION7M (this is us-east-1)",
          "DNSName": "DNS domain name for your CloudFront distribution, Amazon S3 bucket, Elastic Load Balancing load balancer, or another resource record set in this hosted zone. Something like: us-east1.amazonaws.com",
          "EvaluateTargetHealth": false
        }
      }
    }
  ]
}

https://www.youtube.com/watch?v=AAq-DDbFiIE

https://blog.jayway.com/2016/06/02/route53-configuration-elastic-beanstalk/

http://docs.aws.amazon.com/cli/latest/reference/route53/change-resource-record-sets.html

API Blueprint Video Transcript

Slate is an option for generating static documentation for API. You can fork it. You can clone the forked repo. It uses redcarpet for syntax highlighting. The index markdown page shows how you can specify the tabs for the languages, footers and searching. It shows a sample document that you can use as a starting point. You need to run bundle to install the gems. You can then run the server with this command. We can now browse the documentation. You can customize this by replacing the book with your logo. You can see sample code for different languages in the tab. You can see the error codes. We can play with the API using curl in the command line. It also has a search feature. You can also have link to developer signup. API Blueprint is another option. You can document your API using markdown. You can refer the documentation at API blueprint website for the tutorials and reference. Once you document your API using blueprint, you can use aglio to generate the html documentation for your API. The three column layout is appealing. It is easy to browse the documentation and find the details about the request, response, headers and parameters. I will be discussing about this in more detail in the next paid episode.


Api Blueprint Document Generator
Swagger Sinatra Example
Streamlining API with Swagger
API Blueprint Examples

S3 Screenshots

HTTP/1.1 403 Forbidden

x-amz-request-id: 10C5788F4F509396

x-amz-id-2: XfbWddbS3y3W3vA2IXN/CipeuBPbbT0CTpfS502UrUOpu3Doy5KxYnS7XDPcjR1K3yPhq+cnC/k=

Content-Type: application/xml

Transfer-Encoding: chunked

Date: Wed, 19 Jul 2017 22:06:19 GMT

Server: AmazonS3

Curl output will be forbidden.

It took 35 seconds to upload 65 MB file.
It toook 36 seconds to upload 68 MB file.

API Blueprint Screencast Transcript

You can create an account with apiary by using your Github account. I have already created an account. I can now sign in using my Github account. As soon as you login, you will get a sample API to get familiar with tool. We can use this as the starting point for documenting our API. I have customized it for my project. We need to change the polls to the name of our API. We can change the request json to our application needs. Let's link our apiary account with Github. We need to create a repo for our API documentation project. Now we can Authorize apiary. Let's link this project to Github. We can search the repo and connect it. Let's add a dummy file to make this work. Let's commit and start sync. If we reload this page, we can see that sync succeeded. Let's make a change on Github and see the changes on apiary.

We can add MSON markup. We can define a product object and it's attributes. We can commit these changes. We need to remove the white space to fix these errors. We can save and push these changes to Github. We can now see this checkin on github.

We can add a header for products resource. We are going to replace all the JSON to avoid duplication and use MSON. We can declare the attributes of the response. It returns an array of products. The product has an id. We get a syntax error. We can fix it by using an object declaration and providing all the attributes under the object.

We can get a specific product by providing it's id. We need to specify the input parameter. We also specify the response and the status code. We need to provide the attributes of the response. The product object has id as the primary key.

Instead of duplicating the attributes we can use MSON format and specify that the response will be an array of products. Here we can include the product as the response. A product can have many landing pages, we can specify it here.

We need to define the landing page object. We can now install the aglio tool to generate the API docs. Let's get all the files from github. We can now generate the API documentation. We can see the JSON body. We can get all products. We can create a new product.  We can also generate a three column documentation which looks better. We can see all the endpoints, the request and response JSON. Members only access video. Forgot password feature.

References

Aglio
HTTP API Documentation with Blueprint
Using API Blueprint with MSON

Installing ElasticSearch on Mac

java -v
No Java runtime present, requesting install.

Prerequisite: Java 8 (only Oracle’s Java).

Install Java version 1.8.0_73 or later.

Download Java:
https://java.com/en/download/

Go to control panel and click on Java to check the installed version.


$ brew install wget

wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.4.0.zip

$ brew install md5sha1sum

sha1sum elasticsearch-5.4.0.zip
6ca6f8157fe2bb6b9d07ac6c77a96a5ff366e420  elasticsearch-5.4.0.zip


No Java runtime present, requesting install.
Elasticsearch requires at least Java 8 but your Java version from /usr/bin/java does not meet this requirement

Download Java SE Development Kit
http://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html

$ java -version
java version "1.8.0_131"
Java(TM) SE Runtime Environment (build 1.8.0_131-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.131-b11, mixed mode)

You have to install JDK not JRE.

Go to the elasticsearch-5.4.0 directory and run:

$ ./bin/elasticsearch

Check Elastisearch

$ curl -XGET 'localhost:9200/?pretty'
{
  "name" : "hnG_yFT",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "dk03YftTQrO6qH0iLxSkHg",
  "version" : {
    "number" : "5.4.0",
    "build_hash" : "780f8c4",
    "build_date" : "2017-04-28T17:43:27.229Z",
    "build_snapshot" : false,
    "lucene_version" : "6.5.0"
  },
  "tagline" : "You Know, for Search"
}

References

[Install Elasticsearch](https://www.elastic.co/guide/en/elasticsearch/reference/current/zip-targz.html)
[Searchkick Gem](https://github.com/ankane/searchkick)

jmespath screencast notes

$ pip install jmespath-terminal
-bash: pip: command not found
$ sudo easy_install pip
Password:
Searching for pip
Reading https://pypi.python.org/simple/pip/
Best match: pip 9.0.1
Downloading https://pypi.python.org/packages/11/b6/abcb525026a4be042b486df43905d6893fb04f05aac21c32c638e939e447/pip-9.0.1.tar.gz#md5=35f01da33009719497f01a4ba69d63c9
Processing pip-9.0.1.tar.gz
Writing /tmp/easy_install-V8meTp/pip-9.0.1/setup.cfg
Running pip-9.0.1/setup.py -q bdist_egg --dist-dir /tmp/easy_install-V8meTp/pip-9.0.1/egg-dist-tmp-lS7KrD
/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
  warnings.warn(msg)
warning: no previously-included files found matching '.coveragerc'
warning: no previously-included files found matching '.mailmap'
warning: no previously-included files found matching '.travis.yml'
warning: no previously-included files found matching '.landscape.yml'
warning: no previously-included files found matching 'pip/_vendor/Makefile'
warning: no previously-included files found matching 'tox.ini'
warning: no previously-included files found matching 'dev-requirements.txt'
warning: no previously-included files found matching 'appveyor.yml'
no previously-included directories found matching '.github'
no previously-included directories found matching '.travis'
no previously-included directories found matching 'docs/_build'
no previously-included directories found matching 'contrib'
no previously-included directories found matching 'tasks'
no previously-included directories found matching 'tests'
Adding pip 9.0.1 to easy-install.pth file
Installing pip script to /usr/local/bin
Installing pip2.7 script to /usr/local/bin
Installing pip2 script to /usr/local/bin

Installed /Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg
Processing dependencies for pip
Finished processing dependencies for pip

$ pip install jmespath-terminal
Collecting jmespath-terminal
  Downloading jmespath-terminal-0.2.1.tar.gz
Collecting jmespath<=1.0.0,>=0.4.1 (from jmespath-terminal)
  Downloading jmespath-0.9.3-py2.py3-none-any.whl
Collecting Pygments<3 .0="">=2.0 (from jmespath-terminal)
  Downloading Pygments-2.2.0-py2.py3-none-any.whl (841kB)
    100% |████████████████████████████████| 849kB 928kB/s
Collecting urwid==1.2.2 (from jmespath-terminal)
  Downloading urwid-1.2.2.tar.gz (585kB)
    100% |████████████████████████████████| 593kB 1.4MB/s
Installing collected packages: jmespath, Pygments, urwid, jmespath-terminal
Exception:
Traceback (most recent call last):
  File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/basecommand.py", line 215, in main
    status = self.run(options, args)
  File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/commands/install.py", line 342, in run
    prefix=options.prefix_path,
  File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/req/req_set.py", line 784, in install
    **kwargs
  File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/req/req_install.py", line 851, in install
    self.move_wheel_files(self.source_dir, root=root, prefix=prefix)
  File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/req/req_install.py", line 1064, in move_wheel_files
    isolated=self.isolated,
  File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/wheel.py", line 345, in move_wheel_files
    clobber(source, lib_dir, True)
  File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/wheel.py", line 316, in clobber
    ensure_dir(destdir)
  File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/utils/__init__.py", line 83, in ensure_dir
    os.makedirs(path)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/os.py", line 157, in makedirs
    mkdir(name, mode)
OSError: [Errno 13] Permission denied: '/Library/Python/2.7/site-packages/jmespath'
$ sudo pip install jmespath-terminal
The directory '/Users/bparanj/Library/Caches/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/Users/bparanj/Library/Caches/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Collecting jmespath-terminal
  Downloading jmespath-terminal-0.2.1.tar.gz
Collecting jmespath<=1.0.0,>=0.4.1 (from jmespath-terminal)
  Downloading jmespath-0.9.3-py2.py3-none-any.whl
Collecting Pygments<3 .0="">=2.0 (from jmespath-terminal)
  Downloading Pygments-2.2.0-py2.py3-none-any.whl (841kB)
    100% |████████████████████████████████| 849kB 1.3MB/s
Collecting urwid==1.2.2 (from jmespath-terminal)
  Downloading urwid-1.2.2.tar.gz (585kB)
    100% |████████████████████████████████| 593kB 1.8MB/s
Installing collected packages: jmespath, Pygments, urwid, jmespath-terminal
  Running setup.py install for urwid ... done
  Running setup.py install for jmespath-terminal ... done
Successfully installed Pygments-2.2.0 jmespath-0.9.3 jmespath-terminal-0.2.1 urwid-1.2.2

JMESPath Gem
JMESPath Terminal Utility

JMESPath

What is JMESPath ?

JMESPath is a query language for JSON. You can extract and transform elements from a JSON document.

Why JMESPath ?

Web developers deal with JSON a lot. It may be a javascript client or a client that consumes and API. This is an useful tool to have in your troubleshooting toolbox. You will be able to search a JSON document quickly. You can also chain the queries by using the pipe operator to create your desired JSON output. This is more powerful than the simple search capability found in your favorite editor or IDE. Let's go through some of the basics by working through the tutorial found on the JMESPath home page.

Basic Expressions

Let's look at the simplest JMESPath expression, an identifier. An identifier selects a the value for a given key in a JSON object.

Demo 1 with Custom Voice Over

Let's now see how to use a subexpression to return the nested values in a JSON object.

Demo 2 with Custom Voice Over

Let's now look at index expressions. Index expressions allow you to select a specific element in  list. It is similar to array access in common programming languages. Indexing is 0 based.

Demo 3 with Custom Voice Over

You can combine identifiers, sub expressions and index expressions to access JSON elements.

Demo 4

Let's now look at slicing. Slicing allows you to select a contiguous subset of an array. You can specify the starting index and the ending index. The ending index is the first index which you do not want included in the slice. Given an array of integers from 0 to 9, let's select the first half of the array:

Demo 5

This slice result contains the elements 0, 1, 2, 3 and 4. The element at index 5 is not included.

Demo 6

If we want to select the second half of the array, we can use the expression: [5:10]

The above two examples can be shortened. If the start or step value is omitted, it is assumed to be the start of the end of the array. For example:

Demo 7

The general form of a slice is start:stop:step. So far, we've looked at the start:stop form. By default, teh step value is 1, which means to include every element in the range specified by the start and stop value. However, we can use the step value to skip over elements. For example, to select only the even elements from the array.

Demo 8

In this example we are omitting the start and the stop value. So, the start value defaults to 0 and the stop value defaults to 10. In this example ::2 is equivalent to 0:10:2

If the indexing is negative, then the slice is created in reverse order.

Let's now look at projections. A wildcard expression creates a list projection, which is a projection over a JSON array. In this example, we have a JSON document that describes people, and each array element is a JSON object that has a first, last and age key. We can list all the first names in our list using this expression:

Demo 9

If you want to learn more, check out the tutorials section of JMESPath home page. In a real project, you need a command line utility for JMESPath. Let's install the JMESPath Terminal on play with JSON documents from a terminal session. We can install it using pip.

To use JMESPath in your projects, you can use the jmespath gem. The first parameter is the query and the second parameter is the JSON. It also has indifferent access. It also allows you to query JSON documents in your file system.

mysql install

    current directory: /Users/bparanj/.rvm/gems/ruby-2.4.0@r51/gems/mysql2-0.4.6/ext/mysql2
/Users/bparanj/.rvm/rubies/ruby-2.4.0/bin/ruby -r ./siteconf20170510-4006-hvrm81.rb extconf.rb
checking for rb_absint_size()... yes
checking for rb_absint_singlebit_p()... yes
checking for ruby/thread.h... yes
checking for rb_thread_call_without_gvl() in ruby/thread.h... yes
checking for rb_thread_blocking_region()... no
checking for rb_wait_for_single_fd()... yes
checking for rb_hash_dup()... yes
checking for rb_intern3()... yes
checking for rb_big_cmp()... yes
checking for mysql_query() in -lmysqlclient... no
-----
mysql client is missing. You may need to 'brew install mysql' or 'port install mysql', and try again.
-----
*** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of necessary
libraries and/or headers.  Check the mkmf.log file for more details.  You may
need configuration options.

Provided configuration options:
--with-opt-dir
--with-opt-include
--without-opt-include=${opt-dir}/include
--with-opt-lib
--without-opt-lib=${opt-dir}/lib
--with-make-prog
--without-make-prog
--srcdir=.
--curdir
--ruby=/Users/bparanj/.rvm/rubies/ruby-2.4.0/bin/$(RUBY_BASE_NAME)
--with-mysql-dir
--without-mysql-dir
--with-mysql-include
--without-mysql-include=${mysql-dir}/include
--with-mysql-lib
--without-mysql-lib=${mysql-dir}/lib
--with-mysql-config
--without-mysql-config
--with-mysql-dir
--without-mysql-dir
--with-mysql-include
--without-mysql-include=${mysql-dir}/include
--with-mysql-lib
--without-mysql-lib=${mysql-dir}/lib
--with-mysqlclientlib
--without-mysqlclientlib

To see why this extension failed to compile, please check the mkmf.log which can be found here:

  /Users/bparanj/.rvm/gems/ruby-2.4.0@r51/extensions/x86_64-darwin-14/2.4.0/mysql2-0.4.6/mkmf.log

extconf failed, exit code 1

Gem files will remain installed in /Users/bparanj/.rvm/gems/ruby-2.4.0@r51/gems/mysql2-0.4.6 for inspection.
Results logged to /Users/bparanj/.rvm/gems/ruby-2.4.0@r51/extensions/x86_64-darwin-14/2.4.0/mysql2-0.4.6/gem_make.out

An error occurred while installing mysql2 (0.4.6), and Bundler cannot continue.
Make sure that `gem install mysql2 -v '0.4.6'` succeeds before bundling.

To connect run:
    mysql -uroot

To have launchd start mysql now and restart at login:
  brew services start mysql
Or, if you don't want/need a background service you can just run:
  mysql.server start

Test Behavior vs Implementation

You might have heard that in Rails model:

      validates_presence_of :name

can be tested using helpers like:




should_validates_presence_of :name

The problem with this kind of testing is that it is tied to this specific implementation of validation. If you change the validation to use it's own custom object, this test will break. This is not a good idea.

Conversion Points

Conversion Points

Every time your audience is asked to take an action within your sales funnel some users fall off. The key to a highly profitable funnel is to optimize each one of these conversion points.

Let's see which conversion points matter most, how to optimize them, and the 4 main metrics you should track if you want to profit.

• Lead Magnet
• Opt-In
• Email Followups (Open Rates)
• Order Form Conversion
• Upsell

Every single step in the sales process is a conversion point. Every kind of sales funnel has it's own conversion points. Take a look at your existing funnel and identify all the conversion points. Conversion rates in sales page is one of the conversion point in the sales process. But the entire sales process must be optimized. 

Key Metrics

• Cost of Customer Acquisition 
• Life-Time Customer Value      (Triple the initial sale, in two years or so).
• Cost Per Lead 

You can buy PDF template at Graphic River  and use it to create PDF documents.

Possible Rack Cache Issue

I had to remove rack-cache to figure out why articles index page did not display all articles when a user clicked on a tag to get related articles.

First remove the rack-cache gem, delete rack-cache gem in Gemfile and run bundle. To troubleshoot this problem, in production.rb set:

config.assets.compile = true

In your development machine (not production) run:

bundle exec rake assets:precompile returned 1 instead of one of [0]

This gave a clue that there was a reference to rack-cache in application.rb.

rake assets:precompile --trace RAILS_ENV=production
** Invoke assets:precompile (first_time)
** Invoke assets:environment (first_time)
** Execute assets:environment
** Invoke environment (first_time)
** Execute environment
rake aborted!
LoadError: cannot load such file -- rack/cache Be sure to add rack-cache to your Gemfile
/Users/bparanj/.rvm/gems/ruby-2.3.1@lafon/ge


Remove    

config.action_dispatch.rack_cache = true from application.rb

Redeploy the code and from rails console production in the production machine, run:

Rails.cache.clear

to clear the cache. The problem turned out to be not related to rack-cache. At least one of the suspect has been ruled out. I temporarily disabled the find related articles by tag.

Tagging Multiple Models in Rails Notes

```ruby
class TagMultipleModels < ActiveRecord::Migration
  def change
    rename_column :taggings, :episode_id, :taggable_id
    add_column :taggings, :taggable_type, :string, :default => 'Episode'
    remove_foreign_key :taggings, :episodes
  end
end
```

rake db:rollback

StandardError: An error has occurred, all later migrations canceled:

Add a new migration for:     remove_foreign_key :taggings, :episodes

rails g migration remove_foreign_key_for_taggings


ActiveRecord::InvalidForeignKey


Delete teh new migration and create a new article instead of editing an existing record in an invalid state.


SELECT column FROM table WHERE column NOT IN
(SELECT intended_foreign_key FROM another_table)

select taggable_type from taggings where taggable_type NOT IN (select taggable_id from episodes);

Mysql2::Error: Cannot add or update a child row: a foreign key constraint fails


Mysql2::Error: Cannot add or update a child row: a foreign key constraint fails (`chico_development`.`taggings`, CONSTRAINT `fk_rails_18a01188f6` FOREIGN KEY (`taggable_id`) REFERENCES `episodes` (`id`)): INSERT INTO `taggings` (`taggable_type`, `tag_id`, `taggable_id`, `created_at`, `updated_at`) VALUES ('Article', 542, 4752, '2017-06-06 03:14:34', '2017-06-06 03:14:34')


Import data from production. Merge the last two new migrations to one migration:


brew install elasticsearch

==> Using the sandbox
==> Downloading https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.4.1.tar.gz
######################################################################## 100.0%
==> Caveats
Data:    /usr/local/var/elasticsearch/elasticsearch_bparanj/
Logs:    /usr/local/var/log/elasticsearch/elasticsearch_bparanj.log
Plugins: /usr/local/opt/elasticsearch/libexec/plugins/
Config:  /usr/local/etc/elasticsearch/
plugin script: /usr/local/opt/elasticsearch/libexec/bin/elasticsearch-plugin

To have launchd start elasticsearch now and restart at login:
  brew services start elasticsearch
Or, if you don't want/need a background service you can just run:
  elasticsearch
==> Summary
  /usr/local/Cellar/elasticsearch/5.4.1: 100 files, 35.4MB, built in 1 minute 7 seconds


elasticsearch

org.elasticsearch.bootstrap.StartupException: java.lang.IllegalStateException: failed to obtain node locks, tried [[/usr/local/var/elasticsearch/elasticsearch_bparanj]] with lock id [0]; maybe these locations are not writable or multiple nodes were started without increasing [node.max_local_storage_nodes] (was [1])?


$ brew remove elasticsearch
$ sudo rm -rf /usr/local/var/elasticsearch
$ sudo rm -rf /usr/local/etc/elasticsearch

$ brew install elasticsearch

brew install elasticsearch
==> Using the sandbox
==> Downloading https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.4.1.tar.gz
Already downloaded: /Users/bparanj/Library/Caches/Homebrew/elasticsearch-5.4.1.tar.gz
==> Caveats
Data:    /usr/local/var/elasticsearch/elasticsearch_bparanj/
Logs:    /usr/local/var/log/elasticsearch/elasticsearch_bparanj.log
Plugins: /usr/local/opt/elasticsearch/libexec/plugins/
Config:  /usr/local/etc/elasticsearch/
plugin script: /usr/local/opt/elasticsearch/libexec/bin/elasticsearch-plugin

To have launchd start elasticsearch now and restart at login:
  brew services start elasticsearch
Or, if you don't want/need a background service you can just run:
  elasticsearch
==> Summary
/usr/local/Cellar/elasticsearch/5.4.1: 100 files, 35.4MB, built in 12 seconds
$elasticsearch

  Article Store (14.7ms)  {"id":5091,"exception":["Faraday::ConnectionFailed","Couldn't connect to server"]}
Completed 500 Internal Server Error in 50ms (Searchkick: 14.7ms | ActiveRecord: 9.6ms)


Rails.cache.clear if new production import does not show new episodes.