Tuesday, December 29, 2015

Published my first Google App Engine Python Application


This would have been a post if everything went easily. But it was not easy. Got one trouble. Ok let me start the story.

Introduction

Google App Engine (here after GAE) was there for long time at least from 2011 its no more preview. Unfortunately I didn't get a chance to play with it. Now since my family went to India during the winter season, I have enough time to play around.

I never thought it would be easy as publishing web site from Visual Studio to Azure in single click. But app engine is simple as one click publish.

First I ensured that it still has free quota. Because being in the Microsoft shop in my day job who embraces Azure a lot, there is no way I will be hosting an application into Google cloud. So its for my personal use and for personal use I never wanted to pay money. Oh except my domain and hosting space.
https://cloud.google.com/appengine/docs/quotas

First Python application targeting Google App Engine

Then I started with the below basics on Python based GAE development and how to publish a simple Python web app into GAE.
https://cloud.google.com/appengine/docs/python/
https://cloud.google.com/appengine/docs/python/tools/devserver
https://cloud.google.com/appengine/docs/python/gettingstartedpython27/introduction

Then setup the environment. I already had Python 2.7. So installed only the GAE SDK.
https://cloud.google.com/appengine/downloads#Google_App_Engine_SDK_for_Python

Its very simple. Once we install the SDK. It will provide us an option to run our web site locally. Once we are satisfied we can push to app engine ie cloud

The helloworld sample in google help is very preliminary. So googled for little higher startup sample and I got the below link.

http://www.labnol.org/internet/host-website-on-google-app-engine/18801/

In the above link I was able to see some more better sample which deal with html files as well. I took that did some modification in html and tried to run from local app engine server. It worked fine.

Deployment

Then I tried to deploy to real GAE by clicking on the Deploy button in the GAE launcher. It opened the browser and I entered my google credentials. But unfortunately it didn't publish to my app engine. The window stayed as is with the below message

If your browser is on a different machine then exit and re-run this
application with the command-line parameter

  --noauth_local_webserver

Then I started googling and found that it happens sometimes and if we use the command based deployment it will work. I tried that and didn't succeed.

Then googled with error and language Python. Then I got another workaround to a known issue.

If the local development server is running we will not be able to deploy to GAE. I just stopped the local server and deployed. It worked.

The public url is as follows
http://joystestproject.appspot.com/


Tuesday, December 22, 2015

Integrating Facebook comments into our custom application

This is already solved problem or there are many articles out there which help us to integrate FB comments into our apps. I have never tried it myself before though I had given details to other developers on how simple it is. Since nobody came back with any issue I thought its very simple. Still there was some doubt in my mind about its simplicity. That I just cleared when I integrated it to one of my personal app called Online Prayer Book.

We completed our personal prayer book when our son torn the physical book apart and there was no way to get the Malayalam version of that book in United States. When I say we my wife was also part of the development. That's a unique & great experience. Another motivation was to learn Angular.JS as my day job was not allowing me to use any open source technologies.

http://joymononline.in/apps/prayerbook/index.html
The language of prayer is Malayalam.

Below is the link which I used to integrate FB comments integration.
http://www.webascender.com/Blog/ID/269/How-to-Add-Facebook-Comments-to-My-Website#.VhR3QflViko

Tuesday, December 15, 2015

10 years in software industry - What I learnt?

What's this post about

As of  7 Nov 2015, I have completed 10 years in the software industry. I have started as developer support engineer and now working as application architect. Currently in my third company, working places range from Chennai, Kochi in India and New Jersey, US. Worked in products and projects. Lacks pure consulting. Technologies are mainly Microsoft, little bit of Android and HTML,JavaScript & CSS based web.

Just thought of scratching my learning and some advise to other fellow developers. These are solely my views. You may agree or disagree. Sometimes will have entirely different opinion.
  • Programming
    • Design - every programmer should create/have minimum design before typing.
      • Hardest thing in programming is naming things. Spend some time during design at least for public classes, methods etc...Private things we can change at any time to a meaningful name.
      • Do no think when coding. Do not code when thinking.
        • But these activities needs to happen in short cycles. eg: Think and design for 15mins and code for next 15 mins and commit your changes.
      • We can never build a generic generic program. If it was possible world does't had these much programmers.
      • Think about generic mechanism and patterns, if something is repeating 3 times.
      • Design API by first writing client/consumer apps. Better test from 3 different clients/scenarios before finalizing.
    • Problem solving
      • If struck on a problem for 30 mins, take a break. It will clear our bias and help to think out of the box.
      • If the above is not helping explain the situation to some one who may or may not understand software. 90% of time, during the explanation new idea will pop up in our brain. Brave people can try rubber duck debugging also.
    • Defect fixing
      • Fix code by eliminating code not by adding code.
      • More code, more defects.
      • When testing team reports a new failing combination, try to add that in unit tests when fixing.
      • There are some defects which will never get reproduced in development. But had to fix by looking/reviewing the code.
      • Always write down debugging steps, so that you can restart from where you stopped after attending call from wife or girl friend.
    • Code quality
      • There is no perfect code. After spending days to finalize one class, a new person seeing that first can suggest improvements. Need to balance to meet the release time.
      • Do not refactor for the sake of refactoring. Refactor as part of new features or defect fix. But make sure those are different commits to source control.
      • Achieving SOLID is difficult. At least make sure your functions are doing one thing and write pure functions.
      • If possible get the review done via writing unit tests.
      • During day job, often the priority goes to meeting deadlines and developer will be forced to write bad code. So develop personal projects or participate in open source projects to write perfect code where there is no deadline.
      • It is not possible to meet a deadline without any hacks. The quality should be maintained by fixing it immediately after the release. In case, its single delivery, try to fix before completion.
    • Coding practices
      • Assembly language era is over. JavaScript has got many minification utilities. So let's program for fellow developers to understand. They are going to maintain the code in future
      • There was a misunderstanding that writing unreadable, complex code will keep the job safe. But that is not true as the new generation developers are writing code way earlier than us and they are smart to take over. 
      • Keep state in one place. Make 90% of classes / code stateless.
      • Do not keep different scenarios, you thought during coding in your head. Document as test methods. Else after some months you cannot understand why you wrote.
      • Communicate what code does via proper variables,functions, classes and component(dll) names. 
      • No comments for explaining logic. This is because when next person changes the code there is no guarantee that he will update comments.
        • API documentation is fine. Eg: xml comments in .Net
    • Source control, build & deployment.
      • Commit your changes within maximum of 30 mins. If you are not able to complete something after 30 mins, go back to thinking/planning and split the tasks appropriately. First step can be creating the source folder structure / test classes and committing those.
      • Do not comment the code for using in future by uncommenting. Its there in source control. If your company is not using source control quit immediately.
  • Architecture
    • Tough task is to balance between delivery timeline and quality. Should never go to extremes.
    • 90% of the time it is not built. Its evolved.
    • Do not stop coding. If you do so your design will be just boxes to developers.
    • Spend time to validate whether code is following arch. Sometimes change arch to the way code is going.
    • Try to avoid third party libraries as much as possible because each library is adding one more dependency on your plan.
    • Have an idea/capabilities about all the known technologies at high level. Dig deep only when required.
    • Think how architecture is going to be coded, tested, deployed and maintained before finalizing. At any point a developer, tester, infrastructure or support guy may come and ask how to do something in detail.
    • Learn buzz words / technical jargon to talk to different people.
    • Communicate to different level of people in different levels of details. To a client/stakeholder a holistic approach, to business don't use technical words, to managers only about deliverables and developers only in terms of programs.
  • Project management
    • Follow the below principle in Agile
      • Get it done - Demo
      • Get it right - Put into QA
      • Get it optimized - Production and maintenance.
    • Do not plan more than one sprint.
    • Create a new branch in source control start parallel development only when its really necessary.
    • Setup a CI & CD at any cost. Manual build and installation is old story.
      • If there is no test cases, do not push to production via CD.
    • Try to get code freeze at least 1 day during the build & release days.
    • Let the developers fix the hacks during the code freeze. But do not let them check in to main branch on which the build activities are going on.
    • When every project is starting customer will have a fixed goal. Its us who should throw new ideas,improvements towards them and keep continuing the project.
    • Make sure those ideas are business related and towards cost cutting for client to improve their profit. When client's profit increases our profit will also increase.
  • As a team member
    • Ask others to review your code and accept the good parts. Even ask juniors to review your code.
    • Do not just say its not working to your technical lead without a proper description or screenshot of exception or some links saying its defect. They will think that you don't know, didn't do debugging and google.
    • If you are taking a hack or shortcut, add the details with web links as comment in source code. So that future developers will not blame you. Do not keep it in mails, because the new members into your team will not get the mails.
    • Always give suggestions to business analysts requirement documents, proposed user interfaces. Most of the time they may not be knows what are the capabilities of technology.
    • Try to be truthful to testing team. When they come up with inconsistent help them to reproduce by understanding the code.
  • Team management
    • Always have one meeting in the morning and let the team explain what went yesterday and what is the plan for today.
    • Try to reduce the time required for your team to send reports. Get the details from task/project/work item management systems such as TFS.
    • Always look at the code your team is producing. To your superiors its your code.
    • Keep quick access to release dates, number of features and defects total and pending. if you can remember some showstopper/high priority defect ids, you will be a star.
    • Do not let project depend on one person. The show must go on at any cost.
    • Give ownership of tracks/components to single person or teams for fixed time say 6 months. Rotate the members to different teams in every 6 months. But never replace everyone in track in single shot.
  • Quality in software engineering
    • If you are clicking everyday as part of testing app, try to bring automation or look for next company.
  • Enterprise Application Development
    • Most normal things won't work in enterprise. Many things done there doesn't have any common sense. But have to do to keep the project moving.
    • It is very difficult to bring a change in enterprise.
  • Self learning and carrier improvements
    • Read others code.
    • Debug for others. 
      • There are many who can type code. But very less who can debug code.
    • Try to look into third party libraries. If its unmanaged, it will be difficult but in all the other cases its possible to view the source code of third party libraries. Obfuscation makes it little difficult though.
    • Learn one new language every year
    • Read at least one technology book every year.
    • Improve searching/googling skills. A best programmer needs to be a best googler.
    • Never waste your brain memory for language syntax. Google can give it in seconds from internet. Instead free brain to keep the capabilities and think freely.
    • Do not spend much time on learning short living frameworks such as Silverlight. Instead spend time on long living concepts such as http 2. 
    • Try to create and publish reusable libraries. There are very good library management infrastructures now a days such as nuget, npm etc...At least contribute to libraries. If there is no time to fix defects report the defects in the libraries.
    • Learn how things are working end to end. From business analysis, design, coding unit testing, functional testing, business testing, build systems, continuous deployment, application support and how to work with patches.
    • If you want to become a good business analyst you should be knowing what are the developers problems and how developers interpret the requirements document. For that best way is to work as developers. Similarly if you want to be good developers, you should know how QA tests your app. Again better way is to know is to work as QA. QA may not catch all the defects but its caught in application support. So to do better testing you should be knowing how the users use the system and best place is to work in support. To me an ideal career path is Application support->QA(manual + automation)->Development->Business Analyst. Join as support and climb the ladder, If there is no chance to climb in current company change the company.
    • When to leave company
      • Evaluate your activities in your company at least once year other than performance appraisal. If you feel you got only salary in return, leave the company.
      • If you feel you are the ultimate authority of something and everybody depends on you as you are the king, leave the company, because your growth is over there.

Caution JavaScript Ahead - Functions as Values

We have seen about the need for learning Javascript ahead in one of the previous post. Is the development in Javascript easy? Absolutely not. Lets see one simple scenario in this post along with other scenarios posted earlier.

http://joymonscode.blogspot.com/2015/08/multiple-ways-to-iterate-array-in.html

Functions everywhere ie Functional Javascript

Javascript runs on key value pair mechanism. Even functions can be added to as value to a key. Since everything runs based on key value pairs and we can change the value of a key at any time including run-time, the environment gets really dynamic.

Look at the below code.

var mathLib = {};
mathLib.Add = function (n1, n2) {
    return n1 + n2;
};

We can see that when the object was created it has no function named Add. We added it later. Calling the function as below will give is the sum of 2 numbers passed into it.

var sum = mathLib.Add(2, 3);
return alert(sum); //Displays 5

This works great. But think about a big project where 100 developers are working on. Anybody can write code to change the value of Add key / property of mathLib as follows. This can even happen after the real Add function is added. Lets consider the below click event handler

function alter_mathLibAdd_Click() {
    mathLib.Add = function (n1, n2) {
        return n1 * n2
    };
}

The above is absolutely valid in js and it will change the meaning of earlier add function. Now think about your code calling the add method.

var sum = mathLib.Add(2, 3);
return alert(sum);//Displays 6

Really!. Yes this is just an example. These kind of things can happen at any time in our Javascript code, if we don't take development seriously. That being said don't just jump into Javascript without proper understanding. It was kind of OK if we jump into Java or C# without much preparation because language + compiler has so many things to get a low quality developer moving.

The property is kind of called Homoiconicity of language in theory. Real homoiconicity means the program syntax and data structure are same. But here its more to program as data.

http://jsfiddle.net/joygeorgek/Lwqnpqvx/1/

Welcome to the dynamic world of Javascript.

Tuesday, December 8, 2015

Karel source hosted to Github

2 years back, I have created a educational tool called Karel in Javascript. The idea was not at all mine. I just visualized the idea in Javascript only as part of learning HTML5 canvas. The idea was to publish the source in some open source public repository. But unfortunately that missed from my attention.

Recently I could see a repo which is based on my Karel. There the language is changed to Czech and used Jison parser. I really don't know what is that Jison parser. He might have done some work to extract the code from the application page. It will never be easy as forking from Github.

It really made me to publish the code in open source space as soon as possible. In case somebody needs to do something on top of it, they should be able to jump start in couple of clicks.

Here is the repo in Git

https://github.com/joymon/Karel

I am planning to convert Karel to TypeScript. TypeScript really gives a feel that we can develop big maintainable applications in Javascript.

Tuesday, December 1, 2015

Leveraged Evernote in my personal site http://www.JoymonOnline.in

What we need in personal sites

If we look at any personal profile sites, we can see, its bunch of information about a person or group. Information like what they like, what they do? what is their recent advancements etc...

Problems with HTML / ASP.Net only sites

In the first glance we may think that it can be done via simple HTML pages. That is OK for the first release. Once its released there will be changes requests. It seems human nature that they always want to improve. If the change request is too low, it can be managed by manually editing HTML. But if its more manual editing is not a good option. 

Obviously the option is to build dynamic pages and serve. It brings non functional requirements as below
  • A database and its maintenance.
  • Need to get a good web based HTML editor. 
  • Need to implement security
    • May be a hidden login page as I don't want my viewers to see the login option in my personal site.
Or better use a CMS already available and customize it to suit to me.

Problems 

Maintaining database

Database creation and maintenance seems simple in the beginning. But once we do 10-15 releases, it will be nightmare. Mostly we will end up in so any schema modifications and data upgrades.

Making sure there are periodic backups is again a headache. Another thing is if we want to buy decent database hosting space its not cheap.

HTML Editor

There are good web based ie Javascript based HTML editors available in free. But still I need to spend some time selecting which is the best suited for me and integrate the same.

Security

ASP.Net has good security framework. Without coding much we will get a basic site with login page. But we need to always make sure its secure by applying patches. Sometimes we won't have access to the servers especially in shared hosting.

CMS and customization

I first used still using "Personal Web Site template" from Microsoft which is ASP.Net Web Forms. At that time I felt that is simple and will help me to launch site easier. 

Now No way I can easily upgrade to MVC. Slowly I am removing the framework features and making it simple web form application. There is nobody creating free themes for it now.

If I had chosen Joomla or other heavy CMS, I can get free templates now also.

If we consolidate all of the above what I want is
  • A web site. 
    • Design changes less frequently.
  • Should be cheap to host.
  • Sections are pretty static but content changes are frequent.
  • There should be easy way to edit without worrying about HTML syntax

Solution - Outsourcing content for easy editing

The solution which I feel best suited is 
  • Outsource content management.

  • Pull that content to our site and display it.
    • Need to make sure there is API which gives content in HTML.
  • Login to third party portal to change content
The one I found suitable is Evernote. I can keep my site content (text,list,table) in Evernote and they have API to pull the data.

Like any other solution there are pros and cons.

Pros

  • Its free.
  • No need to worry about database maintenance.
  • No worries about security.(Need to remember the Evernote password :))
    • This is trust/belief based. They can secure the contents than I can.
  • Site contents can be simply edited from mobile on the fly.

Cons

  • If Evernote stops free account, need to migrate all the contents to the web site.
    • Need to properly abstract (class EvernoteContentProvider:IContentProvider {})
  • There is monthly quota for free Evernote account. Need to cache the data in site.
    • If the site has huge hits don't go this route.

Updates

2020-06-17 
After using some time JoymonOnline.in stopped using Evernote as Evernote made collaboration in premium paid tier. Currently it stores the data along with the source code in GitHub