Tuesday, February 27, 2018

Swagger API Test UI for easing PoC development

Background

Developing PoCs (Proof of Concepts) or samples to prove some approach or technology is suitable for our project is some every software engineer does as part of their job. Also if something goes wrong with existing systems in production we often create a sample to isolate and reproduce the problem to analyze further. If the role is of Architect, they have to involve more into these research kind of activities. Please note that create these PoCs are slightly different from prototyping as prototype is more towards visualizing the concept in the initial time only.

The goal of PoC if it is to troubleshoot an issue is to create it to the closest way possible to mimic the actual application in trouble. If the issue is production the time will be limited to create app from scratch. When we say closest it should be closest in terms of the technology, threading model, security etc...

Problem

If it is windows application of console applications, it is easy to develop a PoC and run the same way the original application runs. Even if original application is triggered in the context of a different user configured from windows task scheduler or similar, we can still mimic it. But most of the applications are web apps now a days. If we want to reproduce an issue of Web App, it would be difficult if we create a windows console app unless we run the console application using the same service account of app pool which runs the web app. But still the threading model and all will be different. If we want to develop the PoC as web application, it may take some more time than a simple console application which adds delay.

One possible solution

So the best way is to develop WebAPI and host it using the same app pool of original application. Provided there are permissions to host. Suppose we have permissions to host, the next thing is the easiness to test using WebAPI alone. If we have to develop UI for that it would take more time. So what is the solution?

Swashbuckle

Enter Swashbuckle after the long introduction which help us to render UI for the available WebAPI methods. This helps us to test the WebAPIs from a simple UI dynamically generated by it.

Since there are so many links out there which explains how to use it, lets follow the DRY principle. Links below on how to setup Swashbuckle into the WebAPI.

https://github.com/domaindrivendev/Swashbuckle
https://docs.microsoft.com/en-us/aspnet/core/tutorials/web-api-help-pages-using-swagger
http://wmpratt.com/swagger-and-asp-net-web-api-part-1/
https://elanderson.net/2017/10/swagger-and-swashbuckle-with-asp-net-core-2/

There would be so many different views on solving the original problem of creating PoCs. Feel free to comment your technique of rapidly creating PoCs to mimic the real application.

Tuesday, February 20, 2018

Skype v/s Visual Studio - Pair programming via Skype for business meeting

Last week I was doing a pair programming session with a colleague which works from the other part of the globe. The screen sharing was done through 'Skype for Business'. It was about a generic retry mechanism to be used with Azure KeyVault. Not like the retry code which is already available, we have to get the renewed key from Azure KeyVault in certain scenarios.

I was sharing screen. When ever we are in the intense debate and I do some code change the screen sharing gets lost. I continue with my arguments if the change tried was proposed by me and when I ask why you are silent, other guy tells he lost sharing. It happened  4-5 times when we were trying the changes proposed by me. Then I started thinking what my colleague may having thinking about the screen sharing. Will he be thinking that I am intentionally cutting the screen share? Never because we knew each other for long and we know how software from Microsoft works.

When it get frustrated, we decided to investigate on this. If we cannot solve our computer problem how can we solve others problem with computer? Step by step retries couple of times revealed the root cause.

Root cause

Its Ctrl+Shift+S short cut. As a practice from college where the computers were desktops and power may go at anytime, I used to save immediately after typing something. It continuing now also in the era of using laptops or even mobiles for programming. The short cut in Skype for Business to end screen sharing is also Ctrl+Shift+S.

The real solution is to depend on the Ctrl+Shift+B for building which saves all the files. But really its difficult to change.

https://support.office.com/en-us/article/keyboard-shortcuts-in-skype-for-business-42ff538f-67f2-4752-afe8-7169c207f659
https://support.office.com/en-us/article/keyboard-shortcuts-for-skype-for-business-74eda765-5631-4fc1-8aad-cc870115347a

Tuesday, February 13, 2018

Azure @ Enterprise - Moving databases from SQL VM to SQL Azure

Introduction

Enterprise will have so many databases running in its existing systems. If those systems are legacy the databases might have all the legacy features which SQL Azure (The PaaS offering not the SQL VM in Azure) does not support. How to move such databases to SQL Azure as part of Azure adoption. If anyone wonder why the SQL Azure is not backward compatible with standalone SQL Server versions welcome to PaaS. 

Research & Solution

If we google, we can get so many options on how we can move an on-premise database to SQL Azure. Some of those are as below.

After doing good research, the best option found to be the .bacpac mechanism using SQLPackage utility. As we can see in any production databases, the file groups will be all over the place to increase performance. the bacpac mechanism using SQLPackage will eliminate the file groups issue in it's latest versions.

Problems

But it may not be the easy and hurdle free migration road. Below are some issues.

SQLPackage fails on large tables

The SQLPackage.exe has its own timeouts. When there are large tables the timeouts may hit and it will error out. When it error out, there could be a message as follows.

Processing Table '[dbo].[large tables with millions of rows]'.
*** A transport-level error has occurred when receiving results from the server. (provider: TCP Provider, error: 0 - An existing connection was forcibly closed by the remote host.)

Solution

The message tells some network issue not specifically on timeout. But after trial and error this seems to be related to timeouts of SQLPackage.exe utility. It has some params to control the timeout. The usage is as follows

sqlpackage.exe /Action:Import /tsn:tcp:<databaseserver>.database.windows.net /tdn:<database name> /tu:<user> /tp:<password> /sf:<local path to file.bacpac> /p:DatabaseEdition=Premium /p:DatabaseServiceObjective=P15 /p:Storage=File /p:CommandTimeout=0 /TargetTimeout:600

The highlighted are 2 params worked out. The value depends on the size of the database. /p:Storage=File is must when we deal with large databases anyway. It cannot be the other option where memory is used which may drain out quickly.

SQL Azure Server supports only one collation

If the on-premise application is serving globally the collation of the database might have different than the collation supported by the SQL Azure Server.

If anyone is confused on whether there is a SQL Server for PaaS offering, yes there is one. This easily makes us think that there would be real VMs running behind he SQL PaaS offering.

Coming back to the problem, we can have database in SQL Azure with different collation as of SQL Azure instance. But if there are stored procedures which needs to access the system objects, they fail. The default answer would be to change the collation of SQL Azure Server. But unfortunately it is not supported. SQL Azure instance is kind of hard coded to 'SQL_Latin1_General_CP1_CI_AS' collation.  Microsoft has their own reason it seems. But as user what we can do?

Solution

Modify our SQL code to include collation or change the collation of our database to the collation of SQL Azure instance which is 'SQL_Latin1_General_CP1_CI_AS'. It is simple to say change the collation but in enterprise its sequence of approvals especially if the collation is set as standard across multiple applications.

Conclusion

It is nice and good to use Azure. It works well when the application is from scratch and cloud native. But when it comes to migration of existing applications, its nightmare. Sometimes feels like the Azure is not matured for enterprise.

https://msdn.microsoft.com/en-us/library/hh550080%28v=vs.103%29.aspx

https://github.com/Microsoft/vsts-tasks/issues/1441
https://social.msdn.microsoft.com/Forums/en-US/3dd204b3-603d-4c88-9f85-083f69323cd1/sqlpackage-publish-timeout?forum=ssdt
https://docs.microsoft.com/en-us/azure/sql-database/sql-database-connectivity-issues
https://stackoverflow.com/questions/16089321/restore-a-bacpac-file-to-sql-azure-with-large-database-size-sqlpackage-exe
https://blogs.msdn.microsoft.com/azuresqldbsupport/2017/01/31/using-sqlpackage-to-import-or-export-azure-sql-db/