Tuesday, March 21, 2017

GoDaddy to use Github pages

Recently I setup my personal site www.joymononline.in to point to GitHub pages. The domain registration is still with GoDaddy but the hosting space is no more required from GoDaddy. Infact that was eating good amount of cost involved in maintaining the site.

How to configure GoDaddy to use GitHub pages is already a solved problem. There are many sites which explains the same. Below are some.

http://captainwhippet.com/blog/2014/05/11/blog-setup-details.html
http://mycyberuniverse.com/web/configuring-a-godaddy-domain-name-with-github-pages.html


Why this post?

The IP addresess

One of my confusion was about the IP addresses mentioned in those posts. Are those still valid?Some posts are written in 2015. Fortunately those are still valid.

GoDaddy Records

Another place I was little confused was about setting up A records. There were some A records. Instead of deleting all, I just modified to have GitHub IPs.
Also some sites talk about 2 IP addresses to be added via 2 different A records but some talks about only 1. I had updated 2 records and the site got up with GitHub pages in couple of mins.


Tuesday, March 14, 2017

Architecture of JoymonOnline - Serverless

This can be considered as continuation of 3 old posts.


The first post, talks about the architecture of my personal site JoymonOnline.in. Other posts explains about new software architecture trend called Serverless and the conversion of my site towards Serverless architecture. 

This post explains the new architecture.

Architecture of Serverless JoymonOnline.in

Hosting

Hope the diagram is pretty much self explanatory. Its going to be pure client side single page application (SPA). In other words there is no page refresh when user goes to different view. 

The domain is going to be continued with Godaddy. But the hosting will be done inside Github pages, which is free. When we say Github hosting is free, it comes with limitations and quotas. But for my personal site those limits are not going to hit at least for some years :). It clearly tells that for a different person who is really famous the limits will come into play. So Github pages is not the silver bullet for hosting.

Flow

Once the client browser gets the static site(html,js & css) from Github, it becomes active by bootstrapping Angular components. It calls required services present in external systems for data. Most of these external systems are already mentioned in my previous post. After the data retrieval, the JavaScript updates the DOM with the help of AngularJS framework. The Angular JS version is 1.x used and there is a blog post explaining why 1.x is selected when Angular 2 is out.

Future candidates


  • Azure Function - For Github & Blogger API due to anonymous quota limits.

Pros

  • No cost to host web site. Just the domain registration. Most of the services are free.
  • No need to license development tools. For  ASP.Net, Visual Studio community edition don't have much features.

Cons

  • Multiple points of failure - Earlier, if JoymonOnline.in domain was accessible entire site works. But now all the domains of external systems should be accessible from the client machine which is not possible in all places.
  • More dependencies - Has to monitor what changes in external systems and has to maintain accordingly.
  • JavaScript restriction - Some browsers (inside some companies) might be still blocked to download JavaScript and execute.
Note - As of today (14Mar2017), the redirection for the site from Godaddy to Github pages to serve site has not done. But the site is up and running in Github pages joymon.github.io.

Tuesday, March 7, 2017

ConcurrentCounter - Reduce code with Functional programming

The fashion of functional programming

Now a days functional programming already made its come back and every language, companies and developers want to embrace functional programming. When we look at their works we can see it needs to travel more to reach the functional programming world. It was same for object oriented world as well. People used C++ but the code looked like C.

That is not the concern with this post. One of the reasons we here when people adopt FP is that it helps us to achieve things with less code. This is just a scenario where FP can reduce some amount of code.

Scenario - Thread safe manipulation of resources

We are in the world of threads. Knowingly or unknowingly we had to deal with threads. Some languages / technologies already has better ways to deal threading without developer write any explicit line. NodeJS can be an example. But for C# guys like me, has to write programs properly to make sure it works with threads. If all the tasks are ready to run in parallel,  there are no issues. But the reality is not that way. There will be so many critical sections we had to handle to achieve a good working software. We rely heavily on thread synchronization mechanisms to achieve the same. Whenever we deal with those, it is nightmare and there are so many chances to produce duplicate code. That is the place functional programming can help us.

Taking one situation. We have a counter inside a web service.That counter can be considered as a resource just like file or other resources. Counter variable is accessed by multiple service methods, some reads some writes. So the first way comes to our mind would be C# lock (VB.Net SyncLock) keyword. When we implement it there are chances that we may write lock statement in multiple places. Below is a mechanism to avoid that by using functional techniques of C#.
Please note that in real world no one should use this mechanism for integers as far as Interlocked class is there in .Net framework. This is just a sample for understanding purpose.

Wrap the lock statement in fuction and pass the actions into it

The technique here is simple as anyone might have guessed. Just have a wrapper around the lock and inside the locked section execute the real manipulation code which is passed into it as Action<T>. Lets see the code snippet below.
internal class ThreadSafeCounter
{
    private int internalValue;
    private object lockObject = new object();
    public int Value
    {
        get
        {
            return ExecuteWithLock(() => { return internalValue; });
        }
    }
    public ThreadSafeCounter(int initialValue)
    {
        this.internalValue = initialValue;
    }
    public void Increment()
    {
        ExecuteWithLock(() => { internalValue++; });
    }
    public void Decrement()
    {
        ExecuteWithLock(() => { internalValue--; });
    }
    public void Reset()
    {
        ExecuteWithLock(() => { internalValue=0; });
    }
    #region Helpers
    private void ExecuteWithLock(Action action)
    {
        lock (lockObject)
        {
            action();
        }
    }
    private int ExecuteWithLock(Func<int> fun)
    {
        int result = 0;
        ExecuteWithLock(() => result = fun());
        return result;
    }
    #endregion
}
The main functions to notice here are as below.

ExecuteWithLock(Action a)
ExecuteWithLock(Func)

The lock(){} is there only in one function named ExecuteWithLock() and all other functions those wants to execute critical section of code has to call ExecuteWithLock and pass the code as parameter. That code passing as parameter is obtained by the well known Action<T>. Hope there is no need of further explanation. 

Tuesday, February 28, 2017

Puzzle - Pricing engine done for BarclayCard

Working as Software consultant than a full time employment is fun and make us up to date. We can see different architectures, coding practices, deployments and other software practices which will definitely help in case we have plans to start own company in future. But the down side is we had to attend interviews more often and there is no guarantee that when our assignment is over the market will be in good condition. Sometimes we may not get the job in the technology we want.

Recently I was attending many interviews. One of those was BarClay card. After 3-4 rounds of interview, I was asked to code a problem in production quality. It assumes 100% test coverage. Below is the problem.

Problem

An online retail company conducts market research to competitively price their products.Surveyed data contains Product code, Competitor and Price.

The retail company uses a Pricing engine which recommends most frequently occurring price. If multiple prices occur frequently, the least amongst them is chosen.

Products are classified based on parameters like Supply, Demand. Possible values are Low (L), High (H)

If Supply is High and Demand is High, Product is sold at same price as chosen price.
If Supply is Low and Demand is Low, Product is sold at 10 % more than chosen price.
If Supply is Low and Demand is High, Product is sold at 5 % more than chosen price.
If Supply is High and Demand is Low, Product is sold at 5 % less than chosen price.

Prices less than 50% of average price are treated as promotion and not considered.
Prices more than 50% of average price are treated as data errors and not considered.

Input consists of number of products, followed by each Product's supply and demand parameters.

followed by number of surveyed prices, followed by competitor prices.

Output must be recommended price for each product.

Sample inputs and outputs

Input 1:
2
flashdrive H H
ssd L H
5
flashdrive X 1.0
ssd X 10.0
flashdrive Y 0.9
flashdrive Z 1.1
ssd Y 12.5

Output 1:
A 0.9
B 10.5

Input 2:
2
mp3player H H
ssd L L
8
ssd W 11.0
ssd X 12.0
mp3player X 60.0
mp3player Y 20.0
mp3player Z 50.0
ssd V 10.0
ssd Y 11.0
ssd Z 12.0

Output 2:

A 50.0
B 12.1

This has to be completed within 2 days.

Solution

There can be different solutions. Below are some point about the one I did.  

  • For achieving pricing used decorator pattern
  • Coded in C#
  • Tests done using mUnit.
  • Mocks done using Moq library
  • Test coverage using ncover
Code is available in the puzzles repo. It took around 6-8 Hrs for me to complete. Please note that they didn't get back to me after I sent this code. So don't just copy paste the code and use for the same puzzle. May be there is something missing.

Tuesday, February 14, 2017

C# Overloading Action & Func and why {} matters

Basics - Overloading

Let us start with basics of overloading. Will the below code compile?
class SomeClass
{
    void Execute()
    {
 
    }
    int Execute()
    {
        return 10;
    }
}
Certainly not. We know that overloading applies to the arguments. Return type is not considered.

Now look at below code. Will it compile?
internal class SomeClass
{
    void Execute(Action handler)
    {
    }
    int Execute(Func<int> worker)
    {
        return worker() ;
    }
}

Don't spend more time. It will compile. For sure. Will it work as expected? That is tricky question. As of our understanding, we are using strong typed languages to avoid many runtime type related issues. So that we can concentrate on our logical issues more.

Consider the below code and guess what happens
internal class SomeClass
{
    internal void Test()
    {
        this.Execute(() =>  Console.WriteLine("Hi"));
    }
    void Execute(Action handler)
    {
        Console.WriteLine("Inside Execute(Action)");
        handler();
    }
    int Execute(Func<int> worker)
    {
        Console.WriteLine("Inside Execute(Func)");
        return worker() ;
    }
}
SomeClass cls = new SomeClass();
cls.Test();
The output will be
Inside Execute(Action)
hi

The problem

If we call Execute as shown below which function will execute?
internal class SomeClass
{
    int num = 0;
    internal void Test()
    {
        this.Execute(() =>  num++);
    }
    void Execute(Action handler)
    {
        Console.WriteLine("Inside Execute(Action)");
        handler();
    }
    int Execute(Func<int> worker)
    {
        Console.WriteLine("Inside Execute(Func)");
        return worker() ;
    }
}
SomeClass cls = new SomeClass();
cls.Test();
The output will be 
Inside Execute(Func)

Why did this happen? Since the C# language support lambda expressions without explicit return value which is just there it assumed that the intention is to return the incremented value. Sometimes the original intention of developer might be just to increment the num variable. Some conflict happened between the developer and the language.

This is kind of OK scenario. Nothing breaks. Lets see little more complex scenario where it cause trouble.
internal class SomeClas
{
    int value = 0;
    internal void Test()
    {
        this.Execute(() => value++ );
    }
    void Execute(Action handler)
    {
        Console.WriteLine("Inside Execute(Action)");
        handler();
        CleanUp();
    }
 
    int Execute(Func<int> worker)
    {
        Console.WriteLine("Inside Execute(Func)");
        int result = 0;
        Execute(() => result = worker());
        return result;
 
    }
    private void CleanUp()
    {
        Console.WriteLine("CleanUp");
    }
}
The developer wants to do some cleanup after Execute. So he added the CleanUp() inside action and he wanted to avoid CleanUp() in all the other functions. Other functions pass action to so that the CleanUp() will be performed for all the Execute() calls. 

The expected output event with call to Execute(<Func>) is
Inside Execute(Func)
InsideExecute(Action)
Hi

But what happens here is infinite loop. Some clever language geeks might have foreseen thie infinite loop. But normal developer will not.

Fix - Use curly brackets / braces / {}

People think that putting {} is time waste and they avoid it where ever possible. But here it would have saved some time. See the fix below.
internal class SomeClas
{
    int value = 0;
    internal void Test()
    {
        this.Execute(() => value++ );
    }
    void Execute(Action handler)
    {
        Console.WriteLine("Inside Execute(Action)");
        handler();
        CleanUp();
    }
 
    int Execute(Func<int> worker)
    {
        Console.WriteLine("Inside Execute(Func)");
        int result = 0;
        Execute(() => { result = worker(); });
        return result;
 
    }
    private void CleanUp()
    {
        Console.WriteLine("CleanUp");
    }
}

Just adding {} into lambda expressions helps us a lot. If Test() used {} it would have avoided one more function call and could have reached to Execute(Action) directly.

Moral of the story is don't hate braces. Omitting those is not going to save lot of time or decrease size of program.

Considering the API

But please be aware that if we are giving these Execute() methods as APIs there is no guarantee that our customers will use {} always. So we have to overload the functions properly.

Tuesday, January 31, 2017

Serverless JoymonOnline.in - Intermediate update & experience

The Serverless is the new buzzword and it is getting really good attention. It is nothing but an architectural style where developers don't need to worry about servers for deployment, scaling etc... Just write and push the code. It will scale automatically.

In the previous article about Serveless, I had mentioned I am in the process of converting my personal site to Serverless. The decision was purely financial. Why should I give money to GoDaddy for hosting some pages which can be served by GitHub pages for free?

This is just an intermediate update about that effort.

Steps

Below are the steps taken during this effort. It may vary from project to project based on complexity. JoymonOnline.in is just a profile site which don't use any custom modules or handlers.
  1. Avoid server side data rendering
    1. Replace .Net third parties with equivalent JavaScript API
    2. Convert each .aspx page to contain its own Angular application.
    3. Avoid ASP.Net specific skin files and use pure CSS
  2. Client side integration
    1. Replace ASP.Net master page with Angular routing.
    2. Bundling, Minification, CI & CD
  3. Backward compatibility
    1. Links
    2. SEO
Please note that these steps are relevant if the decision is 
  • Not to start a fresh app
  • To ensure that the code base is ready to delivery at any point in time.
If business can keep quite during the migration or the development can be done is separate branch ad later merge, these steps are not required. Just start a new app.

Avoid server side HTML rendering

The current www.JoymonOnline.in site uses ASP.Net web forms technology. In other words it uses server side HTML generation and sends it to the browser. There is no rule that Serverless should not use server side rendering. It can use, provided the server side rendering can scale automatically without doing any extra task. For example a WebAPI service end point can return HTML and it can be rendered at browser with the help of JavaScript. That introduce some kind of client side HTML manipulation. Once we take that decision, there is not much differences between rendering full contents at client side v/s rendering some contents at server and inject it to client side views. So in short Serverless goes hand in hand with client side web apps which we generally call SPA (Single Page application). It can't be done in one step. So sub tasks are as follows.

Replace .Net third party with JS API

If we want to replace the third party .Net API with JS equivalent there are 2 options
  1. Use equivalent JS library
    1. Has to make sure the critical API keys are not exposed. Can do client site authentication or be happy with free quota.
  2. Write a FaaS which uses same .Net API and let it return required data to client.
    1. We have to make sure it is secured and key present only at service and service can only be called from our app.
Coming back to JoymonOnline.in, it uses some third party APIs at server side to get data and render  via ASP.Net. GitHub API, Blogger API are the main third parties. So these calls needs to be done from client machine. For GitHub, it was easy as there is straight JS library. But for Blogger it was little difficult as the Google feed API is discontinued. Finally it worked with blogspot feed url.

But for GitHub, there is a limitation due to the quota for unauthenticated API calls. Since the developer key cannot be exposed to client, it allows only 60 requests / hour / IP address. Since JoymonOnline.in is expecting more than 60 calls from single API it is ok to go with quota. 

Convert each .aspx page to contain its own Angular application

Next step is to convert page by page to Angular. This is to make sure that the application is always in functional condition than like a demolished house before renovation. Convert individual pages to its own angular app and render that app inside the ASPX pages. Let that page be just an angular app host. Noting else. 
In this step we have to identify equivalent third party ASP.Net UI controls, if applicable. For example how to replace Telerik web forms controls with KendoUI.

In case the plan is to first get to ng1 and later to ng2, I would strongly recommend to go with components rather than directives. TypeScript is becoming defacto standard for angular. Use that to avoid debugging time. After this step, we will have ASP.Net application but the functionality working from browser via angular.

Avoid ASP.Net specific skin files and use pure CSS

Next thing to tackle are the styles. If the application is using skin files, we have to get equivalent CSS files. Sometimes we may need to do this along with the previous step. Depends on nature of third parties and our code.

Client side integration

The next big step is to integrate things at client side.

Replace ASP.Net master page with Angular routing.

The main pending thing is to have angular master page and routing. We already have separate components in different apps. Just create new app in index.html and have the routing to load proper components. This steps sound simple but it has many things to do.

Bundling, minification, CI, CD etc...

Now its the time to optimize the site and do integration to build pipeline. Till this point, the expectation is that we are able to use existing build system. Recommended is Webpack for client side activities such as compilation and bundling. CD is always confusing with Continuous Delivery and Continuous Deployment. At least make sure it is Continuous Delivery.

Current, JoymonOnline.in using AppVeyor for CI & CD. Though AppVeyor is mainly intended for Windows development, it supports NodeJS as well. So decided to continue in AppVeyor. At this point the build output will be pushed to \docs folder as staging. GitHub pages recently started supporting serving web pages from \docs folder instead of gh-pages branch.

It is not fun to develop JS based client side app using Visual Studio solution file. After this step we don't need .sln file to work with. Node rules the client side web development world. At some point we have to introduce Node. So why not from the start if it is possible to live with only Node? When it is said 'start', it means the starting of the migrated application. If anyone feels they can live with .sln file without Node, let them continue. But better don't use mix. Comparing Node with .sln is not the right comparison. But the intention is to use Node and its ecosystem such as NPM & package,json etc... 

Backward compatibility

These may be optional depends on scenarios.

Links

This will make sure that the bookmarks saved by users are still valid. The old ASP.Net application could have modified to redirect the .aspx pages to new URLs. But for JoymonOnline.in, this is not done.

SEO

The SEO efforts needs to be put if the application is public facing.

These steps itself needs explanations which will be having contents enough for separate post. At this point, the entire application has converted and available at joymon.github.io . The main pending thing is to change settings in GoDaddy to point JoymonOnline.in records to joymon.github.io.

Tuesday, January 24, 2017

My JavaScript Module experiments - CI & CD

This is second in the series "My JavaScript Module experiments". This post talks about setting up a simple CI & CD pipeline in modularied JavaScript environment using AppVeyor as CI & CD runner.

Other posts in this series below.

My JavaScript Module experiments
2 My JavaScript Module experiments - CI & CD

Introduction

This post directly jumps to setting up CI & CD for modularized JavaScript. Here also Webpack is used as module loader as well as to do other operations such as TypeScript transpilation, minification etc...

AppVeyor & GitHub Pages

AppVeyor & GitHub pages are 2 external services used here. AppVeyor provides free CI & CD service for open source projects and GitHub Pages provides free hosting from Github repositories. Basic knowledge about these services are considered as prerequisites.

AppVeyor is mainly targeted to Microsoft .Net stack and it automatically takes care, if there is a Visual Studio solution file. But we can make it work for Node as well by disabling the default build process. That can be done by simple changes inside appveyor.yml file. AppVeyor supports configuring CI & CD activities either in a appveyor.yml file found in the GitHub repo or through their UI. Here we are using the appveyor.yml  file as its easy to understand. Though is a file publicly accessible file in repo, we can include sensitive information inside it with encryption.

Separation of Concerns - What does what

When we look at the technologies used, we can see those can so same tasks. AppVeyor can run normal commands as well as PowerShell and even start NodeJS which in turn starts Webpack. Using normal commands, PowerShell & Node we can do file copy operations. Webpack which can be started as command can also execute code which does file copy. What is the problem if all the technologies can do same task? The problem is lack of clarity on which component does what.

For small projects, it is fine. We can have AppVeyor scripts and Webpack do file copy. But when we think of larger projects we need to have clear distinction on that should do what. Else it would be difficult to maintain in future.

The simple rule can be to assign the dev related tasks such as compilation and bundling  to Webpack and deployment tasks to the AppVeyor scripts such as pushing to GitHub. Ultimately AppVeyor is ruling the environment. So there should be one AppVeyor command which starts Webpack. Summarized duties as follows.
  • AppVeyor
    • Downloads the source code
    • Install proper Node version and setup environment by installing packages.
      • This include the Webpack NPM package too.
    • Starts Webpack
    • Run tests.
    • Collect the artifact and deploy  
  • Webpack
    • Compilation
    • Do minification / Uglify
    • Create bundle
    • Emit the output to \dist folder. (\dist is just a convention. It can be any folder)

The CI & CD workflow

The CI & CD workflow can be simple. After the transpilation &  bundling the output HTML app which includes html, js, images etc...needs to be pushed to the \docs folder of repository. The GitHub repository can be configured to serve web pages from \docs folder.

Sample

It is easy to explain the but there are high possibility for missing steps. So to make it more understandable, please refer sample repo located at below location

  • It does CI & CD on 'TypeScript-Angular1-Adv folder'.Other folders are showing different Webpack samples.
  • Push the output to the \docs folder in same repo. \Docs folder is configured to GitHub pages to serve from.
  • It is little advanced Angular 1 sample done using TypeScript.
  • Node & NPM used as dev technology.
  • AppVeyor.yml has all the CI & CD steps. The GitHub token is encrypted and kept inside it.
  • Since the aim of that folder is to minimally show the webpack features, at this point there are no tests included.