A blog about my life, development and projects

Transport/Convert SqLite to MS Sql

Happy new year everyone!

So over the last few weeks I needed to import a lot of GIS and geographical data from a .db file into an existing Sql database.

After asking "the oracle" aka Google for an option on what the best approach would be I found a few posts that led nowhere safe.
Some of the options deals with SqLite ODBC drivers then mapping it as a linked server. Normally this would not be a bad idea, but having then to write a bunch of queries to port the data over to my database just seems like too much work. Besides for that, you might not always have access to set up a linked server.

The best option that I found was a tool called "esf database migration toolkit" but it turns out that you have to buy it. With our South African exchange rate it just seems a little steep, otherwise it would have been an excellent tool.

So as any good developer would do, I wrote my own little tool. In just 4 hours I had a tool that can read the schema from my SqLite database, create the structure with the correct data types on the MS Sql Database and then transport the data for me. The tool is not perfect, but it works.

If you want to use the tool for your own project, or just have a peek at my code you can find it on my GitHub https://github.com/TechnoDezi/SqLiteToMSSqlConvert

Hording data and data retention needs

This blog post will be about a topic that I haven't even thought about in my 13+ years of software development and technology. Hoarding - especially the data kind.

Image result for hoarding data

Tonight while eating my dinner I was watching a show on Netflix about hoarders and people collecting junk, which made me cringe. Over the years I have watched many such shows, without giving them much thought.
My house is always clean, free of clutter, I hate paper to start with and everything must have a place, I periodically throw away or sell unwanted stuff that I don't use.
This is not always possible, as you know "life happens", but I do try and because of this I have never given the hoarding topic much time.

Everyone that knows me will tell you that I have a thing against paper, or the use there of. In this day and age with cloud storage and connected everything the use of paper is quickly becoming a thing of the past.
Tonight I realized with a shock that this can easily lead to a different kind of problem, not just in our personal lives but corporate as well.

Lets first look at what is hoarding:

Compulsive hoarding, also known as hoarding disorder, is a pattern of behavior that is characterized by excessive acquisition and an inability or unwillingness to discard large quantities of objects that cover the living areas of the home and cause significant distress or impairment.

So basically hording comes down to collecting junk and not being able to get rid of it. I definitely don't have a problem there, but it triggered a thought about all the different kinds of hoarding, especially in the technology world and I realized that an excessive collection of data can also be considered hoarding.
After a simple search on the net I quickly realized the topic have been debated quite extensively.

Digital hoarding (also known as e-hoarding) is excessive acquisition and reluctance to delete electronic material no longer valuable to the user. The behavior includes the mass storage of digital artifacts and the retainment of unnecessary or irrelevant electronic data. The term is increasingly common in pop culture, used to describe the habitual characteristics of compulsive hoarding, but in cyberspace.

With a horror shock I now know that I have fallen victim to data hoarding, and yes, it happens to all of us. I have a lot of old hard drives with countless backups over the years, even as far back as my school days, data lying on cloud services, data from way back when, and it goes on and on.
I won't classify this as a problem yet, as I haven't yet shown a reluctance to get rid of it, I simple haven't though about my personal data retention policy - we'll get to this in a bit. But the question everyone should ask themselves is "Do you really need that data from back in 2005?".

Hoarding data in business or corporations even now has a name, called Big Data, and people are making a living trying to give meaning to the endless amount of data that everyone, even businesses collect over the years.
It's perfectly understandable to store data because of legislation or local laws, such as storing medical or financial information for a number of years.

As more and more systems, people and businesses becomes connected and start to generate vast amounts of information it becomes more and more pressing to know what data you should keep, what data you need, and the data that is causing clutter.

In our personal lives we are generating so much data on social networks, chatting, texting, emails, digital photography and videos that losing track of it all is a real concern.
I for one definitely did and will start to put measures in place to not only delete data but to organize the data that I need to keep.

Data retention defines the policies of persistent data and records management for meeting legal and business data archival requirements; although sometimes interchangeable.

Now that we know what data retention means, we will need to define what we will store, why and then lastly a plan on how we would clean up our data.

My steps for a data rention policy looks like this:

  • Is the data a temporary record?
  • Does the data primarily consist of intellectual property?
  • Is the data a permanent record?
  • Have I needed or used the data in the last 3 years?
  • Is there a legal or contractual requirement to store the data?

My plan of action to deal with my data problems will be as follows:

  • Sort photos and videos accross cloud services, social media, delete duplicates, organize into albums and consolodate into one service.
  • Look at all hard drives lying around and delete data that I have not used in 3 years, or no need to keep then consolodate the data that I do need or use.
  • Consolodate all IP and code written to VSTS under the respective projects, including the code written for micro controllers and hobby electronics.
  • Sort and store business related data and properly backup or archive a single copy in accordance to contracts.
  • Securely erase data from redundant or old hard drives and physically throw away the drives.
  • Ensure that I have a backup strategy in place that works, for example using the 3-2-1 strategy. This means having 3 total copies of your data, 2 of which are local but on different mediums (read: devices), and at least 1 copy offsite.

Now that I have a plan I can start getting rid of my digital clutter, clean up my life, and get away from this data hoarding thing.

If anyone have interesting stories regarding data hoarding, please do leave me a comment, or send me a message?

Preparing for a .Net Web Api interview

This post follow on one I did earlier this year named "Great Developers, Interviews and Projects".
Yesterday an old colleague of mine sent me a WhatsApp message just before going into and interview for a position relating to Api development specifically on .Net Web Api and asked me what he should know and how to prepare.

Normally Web Api or any api development isn't a position on it's own and generally forms part of a general .Net web developer role, but I guess it can happen that someone is just looking for an Api developer.

My response to this question was to look at the following links as a guideline on questions that could be asked during such an interview:

There are many more sites out there that could help you prepare not only for an Api related interview but anything from Mvc to general .Net development, they are just a Google/Bing search away.

Most important thing I do during and interview is to try and find out how much you know, and if you just remembered answers to questions. I normally ask more than what I interview for, such as design principles in .Net, clean code principles, database questions. General syntax questions of c#. The most difficult thing to do in an interview is to determine how much someone knows and if it's just theory they parrot back.

During an interview it's good to give someone a test, and I think in general this should be expected in any technical interview. The test can either be a physical test where you could expect to write code, or even write code on a white board or paper without the use of internet or visual studio, I prefer the latter, because it shows me that when you are in front of clients pressed to come up with a design, you have the knowledge to demonstrate your plan and structure your thoughts.

In the previous post I mentioned that if you have code to show, or actively contributing to an open source project or even your own personal GitHub project, to me at least, this counts a lot, because you have something tangible to show and in some cases can be more worth than a test during an interview.

The main point is to be well prepared for any interview. Go over the theory relating to the subject matter and ensure you can apply it. Don't stress, it's OK to say you don't know, or you need time to think about it.

New blog platform with more control

Dear readers

I am glad to announce that TechnoDezi have moved to a new blog platform. Same great site, more control.

You might ask why I would move to a new blog platform if the site still looks basically the same. Well the short answer is that the new platform has more control over the posts, the theme is easier to manipulate and I have the ability to extend it with more great features that is yet to come and hopefully bring additional coolness to this blog.

Some of the features are that my posts will be backed up. It allows files to be stored in the Azure cloud as it's fully integrated. The new blog will also at a later date support invoicing where my clients will be able to pay directly via TechnoDezi. This is in a drive to optimize how I do business and create a greater sense of trust.

Then in the months to come I will be starting my YouTube/Video channel, which I promised in April. The new blog platform will enable me to live stream directly on the blog via Azure media services. To those of you that doesn't speak geek, it just means that I will not be going via a 3rd party service, but keep the media and videos directly on my blog.

The Video channel have been delayed until I can find a suitable workshop with enough space to do recordings, but this is still very much on the top of my list.

Great Developers, Interviews and Projects

Over the past year or two I was responsible for conducting a number of interviews and found that there are a lot of developers that turns out to be just clock-punchers.

This post is purely on how I feel, there is no stats or science behind this, just an observation, and I can be wrong about this. Feel free to leave a comment if you disagree with me.

During a lot of the interviews I conducted I found that most of the developers have no idea what they doing, or they are just there for the pay check. Some are getting paid way more than what they should be, and in some cases I could clearly see that people have been promoted purely on the success of the team and not personal achievements.

Let’s start first with how I operate and what technology and development means to me before I continue with how I conduct and interview and what I look for in new candidates.

For me technology has always been a passion, it’s part of who I am and since I can remember I have in some form or another been involved with learning it, mastering it, making it, and owning it. Ever since I started software development I have strived to be the best, to learn as much as I can and always deliver more than what was asked. I am at the point where I started giving back to the community in terms of my blog, GitHub and soon to launch YouTube videos. But I still find that every day I learn something new.

This view I created of how I operate is most likely why I find it difficult to interview other developers. It pains me deeply when I come across someone in the technology industry and software development that just want’s to go to work and go home, no learning, no experimenting no passion. It saddens me deeply.
Software development I think was never supposed to be a “job”, to me it’s a calling, something you do because you have a deep need to be one with a computer, you speak technology better than your native language.

What I normally look for in an interview is the following:

1. Show me the code

If you have a GitHub page, or some kind of code repository or even a showcase of what you have done before, then you already in the lead. I believe looking at what you can or have done is far more valuable than asking random interview questions.

2. Do you have a side project (or 6)

I always look for developers that has some form of side project. It says to me that you are still actively learning and experimenting. Having a side project also means that you are not just going to work and going home, you go the extra mile. I enjoy seeing developers apply new learnings on side projects, because you never get to do it on corporate systems. If you wait for your day job to give you an opportunity to learn, you will never grow fast enough. This also helps you to build a showcase regarding what you are capable off.

3. Lets play the knowledge game

This one is also very important. In this step I try to probe your knowledge to see how well you know the technology, the industry as well as the previous projects you worked on. This is more like a traditional interview, and some of the questions are pretty standard. In this step it’s easy to see if a candidate can articulate what they are doing, I can also quickly see who have been floating on the success of others and if you will be able to sit in on meetings or even drive them. If you can’t even explain a project in an interview I don’t thing you will be able to design a system in front of a client. Having the ability to articulate your knowledge and explain to others your though process is very important in today’s modern teams. We rely on different people and views to come up with the best solution.

4. Culture

Last but not least is company culture. Will the person being interviewed fit into the culture you have or trying to create. By the time you get to this you should by now know a little about the person, and if not switch the interview to some casual talking and learn a bit more about the person you are interviewing. It is very important that the person is a team player, can collaborate and play nice. There is no point in hiring the best person if they can’t work together to solve problems.


The bottom of the story is there are Awesome developers and not so awesome developers. The awesome developers are like needles in a hay stack and very hard to find. You will go through a whole lot of needles and hay stacks before finding the right candidate. Personally I would have loved it if every developer strived to be amazing, but there are place for every one, and every project do need people that can take over long term support, and unfortunately that is where the awesome developers don’t fit in, they get bored with the mundane day-to-day tasks very quickly.