A blog about my life, development and projects

Ionic Native HTTP

About a year or so back I created a mobile application for a client that was based on Ionic and was asked recently to add SSL pinning for security reason. During this I found and interesting behaviour that isn't very well documented and took me a while to discover.

In order to add SSL pinning I had to change the Http service calls from using Angular HTTP to Ionic Native Http. Everything was all good untill I had to make service calls.

But first, what is SSL Pinning: 

By default, when making an SSL connection, the client checks that the server’s certificate:

  • has a verifiable chain of trust back to a trusted (root) certificate
  • matches the requested hostname

What it doesn't do is check if the certificate in question is a specific certificate, namely the one you know your server is using.

Relying on matching certificates between the device's trust store and the remote server opens up a security hole. The device’s trust store can easily be compromised - the user can install unsafe certificates, thus allowing potential man-in-the-middle attacks.

Certificate pinning is the solution to this problem. It means hard-coding the certificate known to be used by the server in the mobile application. The app can then ignore the device’s trust store and rely on its own, and allow only SSL connections to hosts signed with certificates stored inside the application.

Adding SSL Pinning to an Ionic application:

The Ionic documentation is quite limited when it comes to examples. You can find it here https://ionicframework.com/docs/native/http/

Step 1: Install the required packages

$ ionic cordova plugin add cordova-plugin-advanced-http
$ npm install --save @ionic-native/http

Step 2: Add the plugin to the apps modules

import { HTTP } from '@ionic-native/http';

...

@NgModule({
...

providers: [
...
HTTP
...
]
...
})
export class AppModule { }

 

Step 3: Set SSL pining before making the HTTP service call

this._http.setSSLCertMode('pinned');

 

Issue 1: Now that SSL pinning is set, you can make the service call right? Wrong, the 1st issue was to actually specify the SSL certificate to use. The documentation didn't specify where the certificate should go, or what it should look like. I ended up creating a ".cer" file that consists only of the Public key (Please never distribute your private key). This certificate file I added to "/src/certificates" in my project. After this I had to update my Ionic asset copy to copy this file to "www/cerificates". Sure you could just add it directly to this folder, but I wanted it as part of the build in order to easily change it later on.

Now that the certificate is added the http calls started to work as expected. But some of the calls were still failing which leads to Issue 2.

Issue 2: After adding SSL pinning, the cerificate, and updating all my service calls from angular Http to Ionic Native Http some of the service calls were still failing. After some debugging, WireShark traces I discovered that Ionic Native Http by default sends all post requests as "application/x-www-form-urlencoded". This was fine for the basic calls as .Net Web Api can handle these things, even if it's expecting Json Rest calls, but the complex types were failing becouse they weren't parsed corectly.

The solution to this was simple. All I had to do was to set the Data Serializer for Ionic Native Http to Json. I wasted a lot of time on this which could have been solved if it was in the documentation, but here it is now for anyone else if they battle.

this._http.setDataSerializer('json');

 

Transport/Convert SqLite to MS Sql

Happy new year everyone!

So over the last few weeks I needed to import a lot of GIS and geographical data from a .db file into an existing Sql database.

After asking "the oracle" aka Google for an option on what the best approach would be I found a few posts that led nowhere safe.
Some of the options deals with SqLite ODBC drivers then mapping it as a linked server. Normally this would not be a bad idea, but having then to write a bunch of queries to port the data over to my database just seems like too much work. Besides for that, you might not always have access to set up a linked server.

The best option that I found was a tool called "esf database migration toolkit" but it turns out that you have to buy it. With our South African exchange rate it just seems a little steep, otherwise it would have been an excellent tool.

So as any good developer would do, I wrote my own little tool. In just 4 hours I had a tool that can read the schema from my SqLite database, create the structure with the correct data types on the MS Sql Database and then transport the data for me. The tool is not perfect, but it works.

If you want to use the tool for your own project, or just have a peek at my code you can find it on my GitHub https://github.com/TechnoDezi/SqLiteToMSSqlConvert

Preparing for a .Net Web Api interview

This post follow on one I did earlier this year named "Great Developers, Interviews and Projects".
Yesterday an old colleague of mine sent me a WhatsApp message just before going into and interview for a position relating to Api development specifically on .Net Web Api and asked me what he should know and how to prepare.

Normally Web Api or any api development isn't a position on it's own and generally forms part of a general .Net web developer role, but I guess it can happen that someone is just looking for an Api developer.

My response to this question was to look at the following links as a guideline on questions that could be asked during such an interview:

There are many more sites out there that could help you prepare not only for an Api related interview but anything from Mvc to general .Net development, they are just a Google/Bing search away.

Most important thing I do during and interview is to try and find out how much you know, and if you just remembered answers to questions. I normally ask more than what I interview for, such as design principles in .Net, clean code principles, database questions. General syntax questions of c#. The most difficult thing to do in an interview is to determine how much someone knows and if it's just theory they parrot back.

During an interview it's good to give someone a test, and I think in general this should be expected in any technical interview. The test can either be a physical test where you could expect to write code, or even write code on a white board or paper without the use of internet or visual studio, I prefer the latter, because it shows me that when you are in front of clients pressed to come up with a design, you have the knowledge to demonstrate your plan and structure your thoughts.

In the previous post I mentioned that if you have code to show, or actively contributing to an open source project or even your own personal GitHub project, to me at least, this counts a lot, because you have something tangible to show and in some cases can be more worth than a test during an interview.

The main point is to be well prepared for any interview. Go over the theory relating to the subject matter and ensure you can apply it. Don't stress, it's OK to say you don't know, or you need time to think about it.

Great Developers, Interviews and Projects

Over the past year or two I was responsible for conducting a number of interviews and found that there are a lot of developers that turns out to be just clock-punchers.

This post is purely on how I feel, there is no stats or science behind this, just an observation, and I can be wrong about this. Feel free to leave a comment if you disagree with me.

During a lot of the interviews I conducted I found that most of the developers have no idea what they doing, or they are just there for the pay check. Some are getting paid way more than what they should be, and in some cases I could clearly see that people have been promoted purely on the success of the team and not personal achievements.

Let’s start first with how I operate and what technology and development means to me before I continue with how I conduct and interview and what I look for in new candidates.

For me technology has always been a passion, it’s part of who I am and since I can remember I have in some form or another been involved with learning it, mastering it, making it, and owning it. Ever since I started software development I have strived to be the best, to learn as much as I can and always deliver more than what was asked. I am at the point where I started giving back to the community in terms of my blog, GitHub and soon to launch YouTube videos. But I still find that every day I learn something new.

This view I created of how I operate is most likely why I find it difficult to interview other developers. It pains me deeply when I come across someone in the technology industry and software development that just want’s to go to work and go home, no learning, no experimenting no passion. It saddens me deeply.
Software development I think was never supposed to be a “job”, to me it’s a calling, something you do because you have a deep need to be one with a computer, you speak technology better than your native language.

What I normally look for in an interview is the following:

1. Show me the code

If you have a GitHub page, or some kind of code repository or even a showcase of what you have done before, then you already in the lead. I believe looking at what you can or have done is far more valuable than asking random interview questions.

2. Do you have a side project (or 6)

I always look for developers that has some form of side project. It says to me that you are still actively learning and experimenting. Having a side project also means that you are not just going to work and going home, you go the extra mile. I enjoy seeing developers apply new learnings on side projects, because you never get to do it on corporate systems. If you wait for your day job to give you an opportunity to learn, you will never grow fast enough. This also helps you to build a showcase regarding what you are capable off.

3. Lets play the knowledge game

This one is also very important. In this step I try to probe your knowledge to see how well you know the technology, the industry as well as the previous projects you worked on. This is more like a traditional interview, and some of the questions are pretty standard. In this step it’s easy to see if a candidate can articulate what they are doing, I can also quickly see who have been floating on the success of others and if you will be able to sit in on meetings or even drive them. If you can’t even explain a project in an interview I don’t thing you will be able to design a system in front of a client. Having the ability to articulate your knowledge and explain to others your though process is very important in today’s modern teams. We rely on different people and views to come up with the best solution.

4. Culture

Last but not least is company culture. Will the person being interviewed fit into the culture you have or trying to create. By the time you get to this you should by now know a little about the person, and if not switch the interview to some casual talking and learn a bit more about the person you are interviewing. It is very important that the person is a team player, can collaborate and play nice. There is no point in hiring the best person if they can’t work together to solve problems.

 

The bottom of the story is there are Awesome developers and not so awesome developers. The awesome developers are like needles in a hay stack and very hard to find. You will go through a whole lot of needles and hay stacks before finding the right candidate. Personally I would have loved it if every developer strived to be amazing, but there are place for every one, and every project do need people that can take over long term support, and unfortunately that is where the awesome developers don’t fit in, they get bored with the mundane day-to-day tasks very quickly.

Upload multiple files with progress using JavaScript and .Net MVC

Over the past few years web technologies have progressed significantly. With the introduction of HTM5 and ECMAScript 5/6 (JavaScript) we have seen quite a few interesting new features being introduces such as web workers, web sockets, XMLHttpRequest2, Geolocation and the file api.
Todays post is primarily regarding XMLHttpRequest2 and the file api.

I have spent quite a bit of time trying to find good examples of how to implement asynchronous file uploads in javascript and being able to accept it using C# and MVC.
There are numerous examples explaining how to achieve the client side uploads using the file api and then making a XMLHttpRequest, but there are very few examples on how to properly receive those files on the MVC controller side.

One of the best examples of async file uploads is the Blueimp file upload control: https://github.com/blueimp/jQuery-File-Upload

Even though this is a wonderful control with a lot of features, I needed something that is very light weight, reusable and that can integrate into any .Net C# project without having to create a custom handler, it must use any MVC controller action.
Almost all examples for MVC involves create a custom handler, and to me this just feels wrong.

One thing that I also needed was for the control to be a able to provide me with the form variable that you normally would have with a form post.
The control should also not only be async but actually have a form post as a fallback for older browser.
I also needed the ability to dynamically set the allowed file types whenever I use the control.

With the criteria above in mind I set out to build just that and chose to create a Knockout.js model. Yes, knockout is not dead, and for something like this it’s perfect because it gives you model binding while being able to implement it on a single view without major scaffolding. I also used bootstrap for the styling and drop container as it comes with the standard MVC template and it has a progress bar control.

The full solution can be downloaded from GitHub: https://github.com/TechnoDezi/MVCMultipleFileUpload

Step 1 – Client side code

The first part of this control is the Knockout model, this will handle the drag & drop events, as well as all the upload progress events inside a single model that can be bound to any html control. The model is implemented as a single javascript file in order to keep the size down and to share it is also easier. The single model will also take care of tracking multiple files at once and be able to receive feedback after it’s uploaded.

function FileUploadViewModel(uploadUrl, dropBoxID, defaultFileImg, supportedExtentions, dataHeaderObj) {
var self = this;
self.uploadUrl = uploadUrl;
self.dropBoxID = dropBoxID;
self.defaultFileImg = defaultFileImg;
self.supportedExtentions = supportedExtentions;
self.dataHeaderObj = dataHeaderObj;
self.showResults = ko.observable(false);
self.showFileSelect = ko.observable(true);
self.showSubmit = ko.observable(true);
self.fileObj = function (fileName, fileSize, uploadPercentage, messages, showMessages) {
this.fileName = fileName;
this.fileSize = fileSize;
this.imgSrc = ko.observable("");
this.uploadPercentage = ko.observable(uploadPercentage);
this.messages = ko.observable(messages);
this.showMessages = ko.observable(showMessages);
}
self.fileList = ko.observableArray([]);
self.init = function () {
// Check if FileAPI and XmlHttpRequest.upload are supported, so that we can hide the old style input method
if (window.File && window.FileReader && window.FileList && window.Blob && new XMLHttpRequest().upload) {
self.showFileSelect(false);
self.showSubmit(false);
var dropbox = document.getElementById(self.dropBoxID);
// init event handlers
dropbox.addEventListener("dragenter", self.dragEnter, false);
dropbox.addEventListener("dragexit", self.dragExit, false);
dropbox.addEventListener("dragleave", self.dragExit, false);
dropbox.addEventListener("dragover", self.dragOver, false);
dropbox.addEventListener("drop", self.drop, false);
}
}
self.dragEnter = function (evt) {
evt.stopPropagation();
evt.preventDefault();
}
self.dragExit = function (evt) {
evt.stopPropagation();
evt.preventDefault();
$("#" + self.dropBoxID).removeClass("active-dropzone");
}
self.dragOver = function (evt) {
evt.stopPropagation();
evt.preventDefault();
$("#" + self.dropBoxID).addClass("active-dropzone");
}
self.drop = function (evt) {
evt.stopPropagation();
evt.preventDefault();
$("#" + self.dropBoxID).removeClass("active-dropzone");
var files = evt.dataTransfer.files;
var count = files.length;
self.fileList.removeAll();
// Only call the handler if a file was dropped
if (count > 0) {
self.showResults(true);
self.handleFiles(files);
}
else {
self.showResults(false);
}
}
self.handleFiles = function (files) {
for (var i = 0; i < files.length; i++) {
var file = files[i];
var re = /(?:\.([^.]+))?$/;
var extention = re.exec(file.name)[1];
var fileName = file.name;
if (fileName.length > 100) {
fileName = fileName.substring(0, 100);
fileName = fileName + "...";
}
var size = file.size / 1024;
size = Math.round(size * Math.pow(10, 2)) / Math.pow(10, 2);
var fileModel = new self.fileObj(fileName, size + "Kb", "0%", "", false);
self.fileList.push(fileModel);
if ($.inArray(extention, self.supportedExtentions) > -1) {
self.HandleFilePreview(file, fileModel);
this.UploadFile(file, fileModel);
}
else {
var message = "File type not valid for file " + file.name + ".";
fileModel.messages(message);
fileModel.showMessages(true);
}
}
}
self.HandleFilePreview = function (file, fileModel) {
if (file.type.match('^image/')) {
var reader = new FileReader();
// init the reader event handlers
reader.onloadend = function (evt) {
fileModel.imgSrc(evt.target.result);
};
// begin the read operation
reader.readAsDataURL(file);
}
else {
fileModel.imgSrc(self.defaultFileImg);
}
}
self.UploadFile = function (file, fileModel) {
fileModel.uploadPercentage("0%");
var xhr = new XMLHttpRequest();
xhr.upload.addEventListener("progress", function (evt) {
if (evt.lengthComputable) {
var percentageUploaded = parseInt(100 - (evt.loaded / evt.total * 100));
fileModel.uploadPercentage(percentageUploaded + "%");
}
}, false);
// File uploaded
xhr.addEventListener("load", function () {
fileModel.uploadPercentage("100%");
}, false);
// file received/failed
xhr.onreadystatechange = function (e) {
if (xhr.readyState == 4) {
if (xhr.status == 200) {
console.log(xhr);
fileModel.messages(xhr.responseText);
fileModel.showMessages(true);
}
}
};
xhr.open("POST", self.uploadUrl, true);
// Set appropriate headers
xhr.setRequestHeader("Content-Type", "multipart/form-data");
xhr.setRequestHeader("X-File-Name", file.name);
xhr.setRequestHeader("X-File-Size", file.size);
xhr.setRequestHeader("X-File-Type", file.type);
if (self.dataHeaderObj != null && self.dataHeaderObj != "")
{
xhr.setRequestHeader("X-File-Data", self.dataHeaderObj);
}
// Send the file
xhr.send(file);
}
//Load View Model
self.init();
}

Step 2 – The html and model binding

On the html side the model can be bound to any html allowing you to style the upload exactly as you wish. When binding the Knockout model to the html you can also serialize you MVC model to a json object and send it along with every upload.
 
@{
ViewBag.Title = "Upload File";
}
@section scripts
{
<script src="~/Scripts/knockout-3.4.0.js"></script>
<script src="~/Scripts/FileUploadViewModel.js"></script>
<script type="text/javascript">
$(function () {
//var data = @(Html.Raw(Json.Encode(this.Model)));
//var jsonModel = JSON.stringify(data);
var jsonModel = null;
ko.applyBindings(new FileUploadViewModel(
"@Url.Action("UploadFilePost", "Home")",
"dropbox",
"@Url.Content("~/Content/images/file_type_icons_flat_-13.png")",
["jpg", "png"],
jsonModel), document.getElementById("fileUploadContainer"));
});
</script>
}
<div id="fileUploadContainer">
@using (Html.BeginForm("UploadFilePost", "Home", FormMethod.Post, new { @class = "form-horizontal", role = "form", enctype = "multipart/form-data" }))
{
<br />
<div id="dropbox" dropzone="copy f:application/vnd.openxmlformats-officedocument.spreadsheetml.sheet f:application/vnd.ms-excel" class="hero-unit">
<h2 id="droplabel">Drop zone - Drag & Drop your files here</h2>
<p id="dnd-notes">Only jpg and png file types are supported. Once you drop your files in the dropzone, the upload will start.</p>
<input data-bind="visible: showFileSelect" id="fileSelect" type="file" name="fileSelect" />
<p><button data-bind="visible: showSubmit" type="submit" class="btn btn-primary btn-large">Upload</button></p>
</div>
<table class="table table-striped" data-bind="visible: showResults">
<thead>
<tr>
<th></th>
<th>File name</th>
<th>File size</th>
<th>Upload Progress</th>
<th>Message</th>
</tr>
</thead>
<tbody data-bind="foreach: fileList">
<tr>
<td>
<img style="max-height: 80px" data-bind="attr: { src: imgSrc }" alt="preview will display here" />
</td>
<td data-bind="text: fileName"></td>
<td data-bind="text: fileSize"></td>
<td>
<div class="progress progress-info progress-striped">
<div class="progress-bar" data-bind="style: { width: uploadPercentage }"></div>
</div>
</td>
<td>
<div data-bind="visible: showMessages, html: messages">
</div>
</td>
</tr>
</tbody>
</table>
}
</div>

Step 3 – Server side MVC controller

On the server side a couple of things is to note. Firstly there is an upload helper class that can retrieve the file content from the request for either an async post or a normal form post. This helper will build up a model that contains the file content, the file name, size as well as the json model or form fields serialized to json string that can be easily desterilized back into a C# model class. Once the file is retrieved it can be saved to disc, opened, or pushed to a database.

 
Controller:
[HttpPost]
public async Task<string> UploadFilePost()
{
FileUploadHelper handler = new FileUploadHelper();
//Retriev the file from the request
UploadedFile fileObj = handler.GetFileFromRequest(this.Request);
//Write file to disc
System.IO.File.WriteAllBytes(Path.Combine(Server.MapPath("~/App_Data/"), Guid.NewGuid() + Path.GetExtension(fileObj.Filename)), fileObj.Contents);
return "File uploaded";
}
 
File upload handler:
using MVCMultipleFileUpload.Models;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;
namespace MVCMultipleFileUpload.Helpers
{
public class FileUploadHelper
{
public UploadedFile GetFileFromRequest(HttpRequestBase Request)
{
string filename = null;
string fileType = null;
string fileData = null;
byte[] fileContents = null;
if (Request.Files.Count > 0)
{ //we are uploading the old way
var file = Request.Files[0];
fileContents = new byte[file.ContentLength];
file.InputStream.Read(fileContents, 0, file.ContentLength);
fileType = file.ContentType;
filename = file.FileName;
IDictionary<string, object> dict = new Dictionary<string, object>();
foreach (string key in Request.Form.Keys)
{
dict.Add(key, Request.Form.GetValues(key).FirstOrDefault());
}
dynamic dobj = dict.ToExpando();
fileData = Newtonsoft.Json.JsonConvert.SerializeObject(dobj);
}
else if (Request.ContentLength > 0)
{
// Using FileAPI the content is in Request.InputStream!!!!
fileContents = new byte[Request.ContentLength];
Request.InputStream.Read(fileContents, 0, Request.ContentLength);
filename = Request.Headers["X-File-Name"];
fileType = Request.Headers["X-File-Type"];
fileData = Request.Headers["X-File-Data"];
}
return new UploadedFile()
{
Filename = filename,
ContentType = fileType,
FileSize = fileContents != null ? fileContents.Length : 0,
Contents = fileContents,
FileData = fileData
};
}
}
}
 
Model class:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;
namespace MVCMultipleFileUpload.Models
{
public class UploadedFile
{
public int FileSize { get; set; }
public string Filename { get; set; }
public string ContentType { get; set; }
public byte[] Contents { get; set; }
public string FileData { get; set; }
}
}
 

Give credit where credit is due: Many thanks to Valerio Gheri who gave me a running start: https://github.com/vgheri/HTML5Drag-DropExample