Monday, June 24, 2013

Service Bus for Windows Server - Getting started

Background

Recently we got a big web project which will be public facing unlike most of the other projects. The security becomes really important as its dealing with students and as always students are more brilliant in hacking than the professional hackers. So it is decided to isolate the modules even physically at the DB level itself. Wow...Yes it really means the database machines of each module can be switched off when that module is not in use.

The architecture is really nice. But that doesn't really solve all the problems. The modules share some data among them. Since the database or each module is different, some tables or columns needs to be replicated in to other databases, if corresponding module needs same data. It raised the old famous question about syncing.

How do we sync the data between modules as they are physically separate and even can be switched off individually?

I am not directly part of the project but whenever I am in reviews of the project, and asks about the syncing, I hear same answer. "It will be done by the service bus."

What is this service bus? Are you referring to the same Windows Azure Service bus? Are you referring to any back end database technology which will sync the data? Or is that any magic software which will read the developers mind and copy the data from selected table columns of one database and copy to another table in a different database? Almost every time there will be a big silence.

I had tried Windows Azure service bus in the early days of Azure and its mainly for integrating independent systems using messaging and of course we can leverage messaging for syncing data.

When we develop huge systems especially for web like FB or Google, we cannot develop it as single application. It must be split into different independent sub systems and they should work together to serve the customer needs. To achieve smooth integration and communication among these sub systems there should be a strong messaging mechanism. Service Bus for windows is such a strong messaging mechanism.

If we take a logging module which is an essential feature of big systems, it should be built as independent system. So that all the other modules such as front end web servers and back end processing servers can use this logging without much overheads. To get a better view of importance of subsystems, think in the other way where the logging is integrated in web server. In such a situation web server needs to prepare the html page response as it's primary duty after that, it needs to log. The logging taskcan be split into 2. Prepare the log message and write the log. Obviously the time to prepare the log message cannot be avoided as web server is the only one which knows what it did and what to log. But what about the time to write log. If the web server tries to open a file or DB connection and write itself, it will add some time to the overall turn around time of web request. Instead of that what about just publishing a log message to the service bus, so that it can be processed later by the logging sub system? To develop such a messaging system, first let us look at how to get a hands on experience with Service Bus for Windows server

What is ServiceBus

There are already so many definitions available. The general architecture term for these kind of messaging systems is  Enterprise Service Bus. There are commercial as well as open source implementations of ESB such as NServiceBus, Oracle ESB etc...Microsoft implemented the concept of ESB as 'Windows Azure Service Bus' and they released the same product to stand alone servers (can be an Azure VM or privately hosted) named 'Service Bus for Windows Server'. Are the Service Bus implementations for Azure and Windows Server same? No. You can check the comparison here.

Service Bus for Windows uses SQL Server for keeping the messages and other contextual information.

Installing Service Bus for Windows Server in Windows 7 development machine

Service Bus has 2 parts .One is the infrastructure and services and next is the APIs to interact with the Service Bus. The current version is 1.0. You may download both from 

This contains 3 files as of now. We are interested in 'ServiceBus.exe'. Install by double clicking. The installer uses web plat form installer so that it will install all the dependencies as well. You may avoid the above download by using web plat form installer directly for installing service bus.

Successful installation mostly asks for windows restart.After restarting you can see the below folder created.

<Install drive>:\Program Files\Service Bus\1.0 - This the path to service bus APIs dlls.

Also you can locate so many .sql files as well. Those are required to configure the SQL databases needed
for Service Bus operations.

Configuring Service Bus

Next task is configuring the service bus. For that either you can navigate via start menu and start "Service Bus Configuration" program or directly execute the exe located at 

<install drive>:\Program Files\Service Bus\1.0\Microsoft.ServiceBus.ConfigWizard.exe

This will ask for some basic things about Service Bus farm. Those mainly include your windows domain password, a security key etc...It will create the service bus farm and add the computer to that then create the service namespace etc...Once you configure the farm you can see a dialog which tells your computer is already in the farm along with a button for leaving the farm.

Better copy the details and keep in a text file .The details looks like below

Starting
Created and configured Service Bus farm management database.
Created and configured Service Bus gateway database.
Creating default container.
Microsoft.ServiceBus.Commands.SBFarmInfo

Processing completed
Validating input and configuration parameters.
Installing auto-generated certificate.
Granting 'Log on as Service' privilege to the run as account.
Windows Fabric configuration started.
Running Windows Fabric deployment.
Windows Fabric starting.
Service Bus configuration started.
Updating database.
Service Bus services starting.
Updating local registry.
Successfully added this host to the farm.
Microsoft.ServiceBus.Commands.SBFarmInfo

Processing completed
Name: ServiceBusDefaultNamespace
AddressingScheme: Path
CreatedTime: 6/21/2013 1:00:07 PM
IssuerName: ServiceBusDefaultNamespace
IssuerUri: ServiceBusDefaultNamespace
ManageUsers: <**username**>@<**domain**>
DnsEntry:
PrimarySymmetricKey: <**encryped key ending =**>
SecondarySymmetricKey:

Processing completed
Endpoint=sb://<**computer name**>.<**domain**>/ServiceBusDefaultNamespace;StsEndpoint=https://<**computer name**>.<**domain**>:9355/ServiceBusDefaultNamespace;RuntimePort=9354;ManagementPort=9355 

Processing completed


First Service Bus Application

The first application which we are going to see is about messaging in topics and subscriptions world. There are always 2 ways to have our first application. Obviously first one is create ourselves from scratch. Next download applications of others which can be sophisticated or their first application itself. I selected to take existing application from service bus samples project in codeplex. Downloaded the samples zip and started with ..\CSharp\GettingStarted\BrokeredMessaging\MessagingWithTopics on the assumption that I can change the sample easily to fit to the stand alone machine environment.

Error / Challenge - 1 - Terminologies

I would like to call it as challenge rather than error. The Service Bus introduces so many terminologies. Some of those are already existing and related to other technologies but some are specific to service bus.

STS - This is a general term which deals with the service which serves security tokens
Namespace - Groups the service bus features. eg: one namespace can have queues and topics.
Topic - This is service bus specific term.Refers to the category of messages. Messages will be send using this topic.
Subscription - This is a virtual queue create on topic. Copy of each message will be available in all subscriptions of topic.*
Message - The atomic communication packet in the Service Bus which is assiciated to topic.

Error / Challenge - 2 - Service Bus for windows still refer Windows Azure

Got this error when running the sample after changing the ServiceNamespace and IssuerName  

The token provider was unable to provide a security token while accessing 'https://servicebusdefaultnamespace-sb.accesscontrol.windows.net/WRAPv0.9/'. Token provider returned message: 'Error:Code:401:SubCode:T0:Detail:ACS50009:

Seems like there are still some references to Azure existing in the Service Bus code. When MSFT brought the same framework to standalone Windows Servers they might forgot this. 

I changed the code as follows.

Uri serviceUri = ServiceBusEnvironment.CreateServiceUri("sb", ServiceNamespace,"<full machinename>");


But this doesn't solve the issue .Still it refers Azure. The Uri create became "sb://servicebusdefaultnamespace.servicebus.windows.net/<full machine name>/"

Then I went to the extent of hard coding

Uri serviceUri = new Uri(@"sb://servicebusdefaultnamespace.<full machine name>/");


Now interesting error 
"uri provided servicebusdefaultnamespace.<full machine name> does not match Service Bus domain: servicebus.windows.net."

Oh its still in love with Azure. By default its pointing to windows azure namespace. 

As of my understanding there is no coding / API ways to redirect to our Service Bus installation. So relay on the config way. But before that I did a reflection of the Microsoft.ServiceBus.dll to find out the hard coding and I succed in by exploring the constructor of Microsoft.ServiceBus.RelayEnvironment.ConfigSettings class. There the name of config file is hardcoded to servicebus.config. Inspecting the ReadConfigSettings() method in same ConfigSerrings class give insight to the location where we need to put the servicebus.config file.

The file should be either at the application root folder or in the config folder of .Net framework. Obviously this was identified by others and posted in their blogs. My bad google skills. The location is %Windir%\Microsoft.NET\Framework[64]\v4.0.30319\Config

Error / Challenge - 3 - Authentication and token issues

You will get so many network related issues if you try to execute the sample further. Some are below.

"The token provider was unable to provide a security token while accessing 'https://servicebusdefaultnamespace-sb.<full machine name>/WRAPv0.9/'. Token provider returned message: 'The remote name could not be resolved: 'servicebusdefaultnamespace-sb.<full machine name>'.

Where from that <namespace>-sb came? No idea.

You can try giving the STS Url to the token provider. 

TokenProvider tokenProvider1 = 
                TokenProvider.CreateSharedSecretTokenProvider(IssuerName, IssuerKey,new Uri(@"https://<full machine name>:9355/ServiceBusDefaultNamespace/"));

But then you will get

"The token provider service was not avaliable when obtaining a token for 'https://<machine name:9355/ServiceBusDefaultNamespace/WRAPv0.9/"

What is this WRAP?

If you are not familiar, you can try with WindowsTokenProvider as follows
TokenProvider tokenProvider1 = 
                TokenProvider.CreateWindowsTokenProvider(new List<Uri>() { new Uri(@"https://<full machine name>:9355/ServiceBusDefaultNamespace") });
            
Which will end up as
"The token provider was unable to provide a security token while accessing 'https://<full machine name>:9355/ServiceBusDefaultNamespace/$STS/Windows/'. Token provider returned message: '<Error><Code>400</Code><Detail>Cannot resolve a namespace for scope: http://servicebusdefaultnamespace.<full machine name>/IssueTrackingTopic/"

All these issues / exceptions will be raising when the execute the below statement.

if (namespaceManager1.TopicExists("IssueTrackingTopic"))

So better follow the below way which I collected by googling.


string stsUri = string.Format("sb://{0}.{1}:{2}/"Environment.MachineName, domain,port);
string namespaceUri = string.Format("https://{0}.{1}:{2}/{3}/", machineName, domain,port, ServiceNamespace);
 
TokenProvider tokenProvider = TokenProvider.CreateWindowsTokenProvider(new List<Uri>() { new Uri(stsUri) });
NamespaceManager namespaceManager = new NamespaceManager(namespaceUri, tokenProvider);
            
            // Delete if exists
if (namespaceManager.TopicExists("IssueTrackingTopic"))
            {
                namespaceManager.DeleteTopic("IssueTrackingTopic");
            }


Now its working till the namespace manager checking for topic. But if you debug further it will fail at another point which is nothing but during message sending


                myTopicClient.Send(message);

Error message will be
"The token provider was unable to provide a security token while accessing 'https://<full machine name>:9355/$STS/Windows/'. Token provider returned message: '<Error><Code>400</Code><Detail>Cannot resolve a namespace for scope: http://servicebusdefaultnamespace.<full machine name>/IssueTrackingTopic"

This is because the address which we gave during MessagingFactory creation is wrong. 

Correct as below.

string messagingUri = string.Format("sb://{0}.{1}/{2}/", machineName, domain, ServiceNamespace);
MessagingFactory factory = MessagingFactory.Create(messagingUri, tokenProvider); 

During this long exercise, at a bad moment I though of turning off the network connectivity. From then I started getting another error message 


The token provider was unable to provide a security token while accessing 'https://<full machine name>:9355/$STS/Windows/'. Token provider returned message: '<Error><Code>401</Code><Detail>The security token service cannot translate sid 'S-1-5-*1-3**18*34*9-23***35**9-21*****38-8**0' associated to user '<domain>\<user name>'


This because I am using my company laptop and my login is in the company DNS (companydomain\username). This shows that the Service Bus needs constant connection to the domain controller as well.

Service Bus sample walk through

The modified sample can be downloaded from sky drive.
 

The main assembly reference needed for this application is Microsoft.ServiceBus. The location of Microsoft.ServiceBus.dll is as follows.

<Install drive>:\Program Files\Service Bus\1.0\Microsoft.ServiceBus.dll

As I told this is a modified version of ..\CSharp\GettingStarted\BrokeredMessaging\MessagingWithTopics sample which is available in codeplex. This basically has 2 exes.

Working

First one sender.exe, will create a 'Topic' in the Service Bus installed locally. Then add 2 Subscriptions  and send 3 messages. Wait for key stroke. Upon key stroke, it will delete the created topic along with messages. Which means you need to run the receiver.exe when sender is waiting for key stroke.

Second the receiver.exe listen for the messages using 2 subscriptions in the same topic. One subscription just reads the message and other read and delete messages. Messages are same. That's why if we run the receiver again, it will not receive the messages.

Both the exes can be found in ..\binaries folder

Sample Configuration

Main thing you need to give the proper service name space. The default is - ServiceBusDefaultNamespace.
Next is the Port number. The default is 9355.
The windows domain user account, I used is same as my login. So it will automatically take the credentials.If you are using different token provider you need to change the code. So it is advised to run the sample in the same Windows 7 machine where the service bus is configured

Also this sample doesn't cover the scenarios where the Service Bus is installed in other server machine and different clients accessing from different domains with different authentication mechanisms.

Trouble shooting and monitoring Service Bus

As you seen above, the environment is really hard to maintain. There may be situations where the message may disappear or vanishing of subscriptions or topics etc... So its always good to have a general service bus explorer application to monitor the health of the system. Luckily there are 2 tools available to explore the service bus and they are.

Visual Studio integrated service bus explorer
Service Bus explorer - Download

References

Monday, June 17, 2013

IIS 8 in Windows 8 - HTTP Error 500.19 - Internal Server Error when hosting ASP.Net and WCF

Upgrading our .Net applications from one environment to another is always painful. Even if its a upgrade from the Windows operating system version or the upgrade of .Net framework version. Basically it should be done in the development environment first and then in the QA environment. Also before performing upgrade in the development environment the developers should be having enough knowledge about the differences between existing plan form and new plat form. Also they need to cross verify what all features will be affected by the changes and should plan the mitigation.

Being said all those theories, its not practical always to do upgrade after all these analysis in tight scheduled projects. Recently I had to look into such a plan less upgrade of a .net application from Windows 7 to Windows 8 and below is the error which I got when browsing WCF services hosted in IIS8 . The upgrade was done first in QA environment ie they simply installed the MSIs in Windows 8 environment and the dev team is informed when error happened.

HTTP Error 500.19 - Internal Server Error
The requested page cannot be accessed because the related configuration data for the page is invalid.
Detailed Error Information:
Module
   IIS Web Core
Notification
   BeginRequest
Handler
   Not yet determined
Error Code
   0x80070021
Config Error
   This configuration section cannot be used at this path. This happens when the section is locked at a parent level. Locking is either by default (overrideModeDefault="Deny"), or set explicitly by a location tag with overrideMode="Deny" or the legacy allowOverride="false".
Config File
   \\?\C:\Program Files (x86)\...\web.config
Requested URL
   http://localhost:80/.../dataservice.svc
Physical Path
   C:\Program Files (x86)\...\dataservice.svc
Logon Method
   Not yet determined
Logon User
   Not yet determined
Config Source:
  835:     <validation validateIntegratedModeConfiguration="false" />
  836:     <handlers>
  837:       <add name="ReportViewerWebControlHandler" preCondition="integratedMode" verb="*" path="Reserved.ReportViewerWebControl.axd" type="Microsoft.Reporting.WebForms.HttpHandler, Microsoft.ReportViewer.WebForms, Version=10.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" />
More Information:
This error occurs when there is a problem reading the configuration file for the Web server or Web application. In some cases, the event logs may contain more information about what caused this error.


Normally this application is installed in pre configured OS images as it requires so many prerequisites. In this case the image for Windows 8 was not created which means all the prerequisites and settings needs to be done manually.

Resolution

This specific error routes to the locked HttpHandlers. To get more idea about locking features we should know about IIS feature delegation. We need to go IIS8 manager and point the sever node and select feature delegation.There we could see that the http handerls are locked. Allowing edit option will solve this issue. 

It didn't actually solved our issue instead it showed subsequent issue. In that Win 8machine ASP.Net and WCF was not registered with IIS 8 :-(

Monday, June 10, 2013

"The literal Octal 08 (digit 8) of type int is out of range" Java compiler error

This error came when my wife was doing simple programming in Java using Eclipse IDE. She was trying to do 'number to words conversion' where hard coded int input = 8; can be converted to "Eight" but not int input = 08;

It clearly says the compiler is treating this as octal literal .But this is really new to me as I am more working with C# & VB.Net which don't have this feature. After a google we were able to find the list of supported literals in Java.

http://en.wikibooks.org/wiki/Java_Programming/Literals

Leading 0 - Ocatal eg: 010 -> 8
Leaxing 0x - Hex eg: 0xA -> 10
Leading 0B - Binary eg: 0B11 -> 3 (Supported in latest Java versions >=7)

Monday, June 3, 2013

Online Karel Simulator based on HTML5 & Javascript

Karel is an educational programming language for beginners, created by Richard E. Pattis in his book Karel The Robot: A Gentle Introduction to the Art of Programming @wiki
If you are not familiar with what is Karel in programming, please read this wiki page. As I mentioned in one of my previous blog post, I got to know about Karel, while preparing for a session of Programming for non programmers. It's such a simple language that, anybody can teach programming or use for self study. Universities such as stanford are using this simple language to teach programming methodology to it's students. If you studied / studying in new generation Indian engineering colleges, sorry you don't get this easy option of studying programming. I don't want to use the term 'self financing colleges' where your teachers will be your seniors who just passed in the previous year. I am sure on this, because I studied in this type of college, where I had seen people repeatedly saying 'public static void main...public static void main' during the lab exam days.

Whatever it is, this post is written to not to describe about the colleges, but to talk about Karel language, and a small HTML5 based Karel simulator, I wrote in the javascript language to execute Karel programs. The story which lead me to develop this simulator is as follows.
  • I found the language very simple & interesting to teach programming.
  • The main attraction is the way, we can teach programming to non programmers by simply saying we are controlling a Robot named Karel by some instructions.
  • I could see some interesting simulators where an animated character Karel, is acting as per the program execution.
  • None of those simulators are easy to set up in our environment.
    • Either they are written in Java or other language which is not easy to set up in my environment.
    • Got one .Net simulator using Visual J++, and that story I explained in previous post
  • No HTML JS based simulators, which can be easily set up as browsing a web site.
  • After more google, I decided to create the first one myself.

Karel the robot learns javascript

If you go to the Stanford page, you can see "Karel the robot learns Java". I just borrowed that and changed it to javascript, on a belief that, they will not file patent case on me. This means the simulator is not supposed to run the original Karel language. It can run the Javascript, which will take effect on the Karel standing at the right side simulation area.

What Karel language supports

There are fundamental 2 things which Karel can do.They are
  • turnLeft
  • move (move one cell in the direction which Karel facing at the moment)
This turns to 2 methods in the Javascript. These methods can be used to write the program in conjunction with the other javascript language features such as if, loops, methods etc...If you want to turn the Karel to right direction you need to issue 3 turnLeft instructions or write a turnRight method which contains these 3 instructions or a loop which does the same.

Javascript function to Turn Karel right


//Program to turn Karel right
turnRight();
function turnRight(){
    turnLeft();
    turnLeft();
    turnLeft();
}

How to work with Karel simulator

Write the javascript program at the right side text area and click on "Run program" button at the bottom of text area. You can see the Karel is obeying your instructions.

Want to try HTML 5 JS Karel simulator 

I have hosted it in my own website http://joymononline.in . URL link is 

Source code of Karel simulator

If you have experience in Javascript you might have guessed the core technique behind the simulator. It's the same eval() function in Javascript. There are 2 global functions turnLeft() and move() written and those functions are being invoked from the user input program by using eval method.

I will upload the source later to one public repository.