Isn’t it easy to mask your data with Dynamic data Masking #5

Data security is always one of the important points which can not be ignored. Nowadays if you are working for any specific domain like Banking or Healthcare then there are a lot of compliance rules which you have to follow.

Data Masking is one of the best ways to help you to secure your sensitive data by a dynamic mask encryption.

This is one of the best features of SQL SERVER 2016 which I personally like most.

With the help of Dynamic Data Masking, you are just applying a mask to your sensitive data.  for example, if your system is storing SSN data then it should be visible to privileged or we can say authorized user only.

Dynamic Data Masking has following features:-

1) It masked the Sensitive data.

2) There will be no impact on functions & Stored Procedures and other SQL statement after applying this.

3) Applying the Data Masking is super easy.

4) You can allow any database user/role to see unmasked data by just simple Grant & Revoke Statement .

5) Data is not physically changed.

6) It is just on the fly obfuscation of data query result .

7) It is just  a T-SQL command with basic syntax.

Now , let us understand how to implement it.

Data masking implementation is very easy and below is the syntax for it.

Masksyntax1

Here, if you see the syntax is very simple the only new thing is MASKED and with (function=function name) only.

The function is nothing but the way to mask the data. SQL SERVER 2016 has following  different functions to mask the data

1) Default() function:- This is basic masking with the help of this function you can easily mask any field.

for example, your first name or last name field can be masked like XXXX etc.

2) Email() function :- If your column is email type or you we can say if you store Email in your column then you should use the Email() function for masking.

for example, your email can be mask like  RXXXX@XXXX.com

3) Partial () function:- With the help of this function you can mask specific data length and exclude some part of data from masking logic. for example, 123-4567-789 is your phone number then with partial masking feature you can mask like 12X-XXXX-7XX.

4) Random() function – By the name it is clear that you can mask the data with any random number range we will see more below in the hands on.

Remove Masking :- This is also possible that you applied a masking to a column and later on you don’t want that masking. So , don’t worry it very easy to remove masking from a column. below is the syntax for same.

DropMask

Now, let’s understand this by an example.

In the example we are using a new database “SecureDataMask” in this database we are creating a tblSecureEmployee as shown in below figure.

Create_Table_Secure_Employee

Now, in this table, we are inserting couple of data for testing as shown below

Indiandotnet_Insert_Default_Row

Now we are applying different masking on this table’s column

1) Default Masking : In the table, we are applying default masking on LastName

Indiandotnet_Default_Masking

2) Email Masking :- In the table, we are going to apply Email masking to email column below is the syntax for it.

Indiandotnet_Apply_Email_Masking

3) Partial Masking:- For SSN we are going to apply custom masking. below is the syntax for same. Here as we aware that SSN is 11 characters long in our database. we applied the partial masking to show first two & last two characters in original value and rest other in the mask.

Indiandotnet_Partial_Masking_SQL_SERVER_2016

4) Random Number Masking :-  In our table, we are going to apply Random number masking to Securepin column as shown below.

Indiandotnet_Random_Number_Masking_SQL_SERVER_2016

Here, so far we are done with all the masking now.  let me run the select statement to test it.

Indiandotnet_Select_statement

If you see the data is still in the original state because I logged in using  privilege account “SA”. now, to test the masking let me create a new user account.

Indiandotnet_Create_Login_User_SQL_SERVER_

After creating the account we are trying to log-in with a new account as shown in below screen.

Indiandotnet_Login_With_New_User

After our successful log in, we will run the select statement on same database’s table as we did earlier. If you see below snap you will find that we got masked data for LastName, Email, SSN, and securePin.

Indiandotnet_Masked_Data_With_Less_Preivileged_account

Now, it might be a rare case but suppose you want to remove the mask from any column on which you applied masking then don’t worry it is super easy.

Suppose, from the same table we don’t want mask on the LastName then below is the syntax for same.

Indiandotnet_Removing_Mask_from_column_Sqlserver2016

Now, let me run the same select statement seeMask_user. You will find the Last Name is unmasked now.

Indiandotnet_Last_Name_Visible_

From above few changes you can secure your data via Dynamic masking and as mentioned above there will be no impact on your existing function ,stored procedure because data is not physically changed.

I hope you may like this feature.   Please, share your input for same.

Enjoy !!

RJ

Surprise ! Visual Studio 2017

Isn’t it surprising ? Microsoft announced Visual studio 2017 RC on 16-Nov-2016.

You can get download the RC version from following link

Download Link

Now , your question will be What’s new in Visual studio 2017 ?

To know more about Visual Studio 2017 check the release note of Visual studio 2017 with following link

Release note Link

I am pretty much sure you will enjoy this latest release.

Please, share your thought for same.

RJ

The Evolution of DATEDIFFBIG in SQL SERVER 2016 #4

In the series of SQL SERVER 2016, this is a new post. in this post, we will discuss DATEDIFF_BIG and how it is helpful.

So, before jumping into directly in technical details, we all know that time is very important and every second valuable and countable but sometimes every microsecond & nanosecond is also countable Smile . For such operations in which every microsecond & nanosecond is countable, we can use DATEDIFF_BIG function.

As you aware the BIGINT range is from -9,223,372,036,854,775,808 to 9,223,372,036,854,775,807.  Here if any difference (Micro & Nano) second is out of the the mentioned range then DATEDIFF returns that value else return error(Obviously).

Below is the basic syntax if DATEDIFF_BIG although it is similar to DATEDIFF. We can say it is a extended version of DATEDIFF.

DATEDIFF_BIG( datePart, start Date, End date)

The value of datePart is same like DATEDIFF function.

For example if you want to collect millisecond difference then use ms, microsecond then mcs and for nanosecond ns.

As per the MSDN   for the Millisecond, the maximum difference between start date & end date is 24 days, 20 hours, 21 minutes and 23,647 seconds. For Second, the maximum difference is  68 years.  

Now, let see why this DATEDIFF_BIG introduced so, I am running a DATEDIFF  function in SQL SERVER 2012 and see what we get after running that query.

 

DATEDIFF_Function_Issue_Indiandotnet

 

You can see in above query we got an error of overflow.

Now, we are calculating the same difference from DATEDIFF_BIG in SQL SERVER 2016. See, below snap for same.

 

DATEDIFF_BIG_FUNCTION_INDIANDOTNET

 

Isn’t it great ? Although, I am scarred with those applications who calculate milliseconds Sad smile.

Anyways, it is good to know feature.

Do provide your feedback for the post it is very valuable for us.

RJ !!!

Here Comes New Idea of Split String in SQL SERVER 2016 #3

In the Series of SQL SERVER 2016, this is another post. Before Jumping in detail just think if you have a comma or other separator string and if you have to split it by separator field then for such task  in previous SQL SERVER versions either you will write a function which split the string and return desire values in a column  or

you will use XML function or  might be different custom functions.

Let me explain this with below example. Suppose you have a string like below

DECLARE @FriendList AS VARCHAR(1000)

SET @FriendList =’Ravi,Suyash,Vaibhav,Shyam,Pankaj,Rajul,Javed’

 

Now you want output like below

output_of

 

Then in such cases, you will  follow 2 approaches (their might be other as well)

Approach 1:- Write  a function like below  and use it.

Traditional_way_Split_String_Indiandotnet

And once this function is created you can use like below

use_of_Split_Function_in_SQL_Indiandotnet

Approach 2 :- You can use XML option in SQL SERVER as  shown in below

String_split_by_XML_SQL_SERVER

So, the good news is now in SQL SERVER 2016 you don’t need to write  so many lines to split any string. In SQL SERVER 2016 a new string function is Introduced which is

STRING_SPLIT

The use of this function is very easy and below is the syntax

STRING_SPLIT (string, separator)

Now, let me show you same output using STRING_SPLIT function

string_Split_Function_sql_server_2016_Indiandotnet

Isn’t it easy ?

I hope you will like this easy way to split the string.

Provide your feedback.

RJ !!!

Compress & Decompress–new Feature in SQL SERVER 2016 #2

This is another article in the series of SQL SERVER 2016 Journey . I am pretty much sure you might aware of Gzip Compression algorithm. If not then try  this link.

 

So, SQL SERVER 2016 introduce this two awesome functions for Compress & Decompress the data.

Before SQL SERVER 2016 version we have data compression feature like Page & Row compression (check Previous post for it Link )which is different then this column value compression.

 

In SQL SERVER 2016 Compress function,  data compression is done via GZIP algorithm and return VARBINARY(MAX).

 

Below is the simple syntax of Compress function

 

Compress (Expression)

Here Expression can be nvarchar(n), nvarchar(max), varchar(n), varchar(max), varbinary(n), varbinary(max), char(n), nchar(n), or binary(n)

 

Decompress function is just opposite of  compress function. It is used to decompress the value of VARBINARY which is converted using Compress function. The only tweak is you need to cast the output of Decompress function  in specific data type to make it readable (if using varchar ,nvarchar compression) .

 

below is the simple syntax of Decompress
Decompress (Compressed string)

 

Let’s understand this via an example as shown below .

 

Indiandotnet_Compress_Decompress_Feature_SQL

In this example I have taken 3 tables with exact same schema & data

 

  1. 1) IndiandotnetFriends
  2. 2) IndiandotnetFriends_Compress
  3. 3) IndiandotneFriends_Decompress

 

You can see  snap in which we are inserting same data.

As the name suggested in first table normal data from Adventureworks’s person table.

In second table we are inserting compressed value of first Name  and in 3rd table we are inserting decompress value of First Name from the Compressed table.

Now, let’s check compress  & decompress table data

Check_Compressed_Decompress_Datais

 

Now, Your might thinking that the output of both compress and decompress is not readable.

So you are right to make data readable of Decompress table we need to type cast.

See below snap for same.

 

Decompressed_value

 

Till now we know how to use this Compress & Decompress function. Now, let me share the benefit of using Compress. if you see below snap you will find that data length of compress is comparatively less than normal and decompressed data length .

 

DataLength_Indiandotnet

 

Obviously, compression helps you somewhere in the overall performance of your application.

The good point is  you can pass the compress data to your .net application and decompress using GzipStream as well.

 

The only thing which we need to take care is type casting. Suppose your base column which compressed is VARCHAR then you need to typecast again in VARCHAR.

 

Now, next question is where we can use this functions. So,  we can use in compressing large object like binary data in which we save jpg, pdf , word document etc..

 

I hope you will be excited in using this function.

 

Please, share your input.
RJ!

DROP IF Exists- A cool feature in SQL SERVER 2016

In the Series of SQL SERVER 2016 journey, this is our new article. In this article, we are sharing a new cool feature which introduced in SQL SERVER 2016 which is DROP IF EXISTS (DIE) .

In our development many times it happens that we need to drop a table and as a best practice we write the following syntax as shown in below figure

TraditionalWay_Indiandotnet

Now, in SQL SERVER 2016 the same task is super easy. You can write the following syntax to drop the table object

DROP_TABLE_IF_EXISTS_INDIANDOTNET

DROP TABLE IF EXISTS TABLENAME

The best part is if suppose the object does not exist then  here will be no error execution will continue.

Let me share one more example of Dropping a stored procedure.

DROP_PROCEDURE_IF_EXISTS_INDIANDOTNET 

Similar, way we can write for following data objects and with the following syntax

Procedure:-
DROP PROCEDURE IF EXISTS Procedure Name

Assembly:-
DROP ASSEMBLY IF EXISTS Assembly Name

ROLENAME :-
DROP ROLE IF EXISTS ROLENAME

TRIGGER :-
DROP TRIGGER IF EXISTS Trigger Name

VIEW:-
DROP VIEW IF EXISTS View Name

RULE:-
DROP RULE IF EXISTS RULENAME

Type:-
DROP TYPE IF EXISTS Type Name

Database:-
DROP DATABASE IF EXISTS Database Name

Schema:-
DROP SCHEMA IF EXISTS Schema Name

User:-
DROP USER IF EXISTS Username

SECURITY POLICY:-
DROP SECURITY POLICY IF EXISTS Policy Name

View :-
DROP VIEW IF EXISTS View Name

FUNCTION:-
DROP FUNCTION IF EXISTS Function Name

SEQUENCE:-
DROP SEQUENCE IF EXISTS Sequence Name

INDEX :-
DROP INDEX IF EXISTS Index Name ON Table Name

Synonym:-
DROP SYNONYM IF EXISTS Synonym Name

I like this feature I am sure you will also like this.

Please, do share your feedback for blog post.

Enjoy !!

Unbelievable SQL SERVER 2016 Feature you should aware.

 

Although, I know I am bit late to share this thing on our blog but it says in Indian proverb “Der aai durust aai” means it’s OK you came late but you came that is more important.

Anyways, so you all might aware that Microsoft launched SQL SERVER 2016 officially in June 2016.

You can download the SQL Server 2016 via Link.

Obviously, this is a new revolution in SQL SERVER series. You will find many great features in this version.

This post is beginning to explore all those great features and we will do deep dive in all those features. In this, post we briefly introducing those features. so, without wasting time let me share a brief introduction.

 

1) JSON in SQL SERVER :-

Is this surprising to you ? Obviously, yes. As you might aware that most of the NO SQL database use either JSON or XML. As XML feature already exists in SQL SERVER so this was time for JSON. You can play with JSON in SQL SERVER 2016.

 

2)Always Encrypted :-

If you talk about security this one the best feature. Now, you are thinking what it means. So, It means that the data in the SQL SERVER reside always in encrypted format and SQL server can perform the operation on the encrypted data without decrypting it. The encryption key can be exist in some other system. With this, feature you can secure you ensure that your data is secure from the person like DBA / Developer as well. These guys also can’t see the actual data. Isn’t it neat ?

 

3) Row Level Security :-

This is another interesting feature which helpful especially to the developers  who needs to write extra code to check this. Let me explain this with an example suppose you have a sales team who do market research and you want to restrict that each sales manager can see only those data which entered by him only in such cases you don’t need to write specific condition in your code. It can be achieved by Row Level Security.

 

4) “R” in SQL SERVER :-

For the data scientist, it is a great NEWS. As Resolution Analytics is purchased by Microsoft and It is incorporated in SQL SERVER. You can run R analysis query in SQL Server.

 

5) Temporal Table:-

The Temporal table holds the old version of ROWS of a table. It means that it maintain a copy of the old rows in the table whenever there is an update on the main table.

 

6) PolyBase :-

With the help of this feature, you can access data which exist in Azure Blob or Hadoop cluster using the same SQL server. In the nutshell, we can say this is the technology which combines both relational & non-relational database in a single umbrella which is SQL SERVER. You can run the query directly on external data like Hadoop or Azure blob storage.

 

7) Stretch Database:-

I am pretty much sure by the name you can guess this feature. So, with the Stretch database you have can store your part of the data in the cloud which depends upon your need. You can say most recent transactional data you can store in your local environment and other old data you can store in Azure.

 

8)Query store :-

Another interesting feature to help you in identifying  performance drag using Query store. When you enable this feature it automatically captures a history of queries , plans,  and statics and retain them for review and resolve the performance issues.

 

9) Mobile report:-

As mentioned earlier this is the revolution in SQL SERVER 2016. In SSRS there are many important changes introduced. Now we can import Power BI report in SSRS and apart from this you can create a mobile report which you can run on Mobile.

 

Now, we started officially SQL SERVER 2016 tutorial series.

 

Moving forward we are going to discuss all these features in details and also the couple of new things which introduced in SQL SERVER 2016.

 

enjoy.

Happy VijayDashmi.

The Dummies’ Guide to How to solve Problems with Extension Method

I am pretty much sure that, You have used Extension method in your program. Did you think to create your custom Extension method ?

If not then this article help you to create your own Extension method step by step.

Before jumping to direct practice let’s understand first what is extension method. So, an Extension method is created to simplify the code. It is a static method which represent as instance method. It accepts “this” keyword as a parameter.

If you see below example image you are not going to surprise as you have used this type of method.

extension_Method_1-1

Now, let me tell you more about the Extension method. As you now aware it is a static method & it accept “this” as a parameter so just need to add a “.” dot symbol with variable and all the existing method will be appear.

The question come to our mind how to create our own Extension method. So, it is simple and easy. We need to create a static class first and then that static class we can create our own Extension method.

extension_method2-1

Once you added a class in your solution specify the class as a static class and a static method in the class. In below example I am using static method with name  “MyStringCharacterLength”  if you see the below snap you will find in the parameter we are using this which is must for an extension method. We are not doing any fancy with this method it is a simple logic of calculating the length of the string.

Extension_Method_3

Now once the extension method is created we can use this method using by a “.” dot symbol. Isn’t it easy let me show you the code.

extension_Method_4

When we run this code it will return the length of the string which provided as a input.

I hope a basic now you have a basic understanding of extension method. You can create your various method to use this and reduce your efforts in cases where you have to write similar logic again and again.

I recommend below site from where you can download existing Extension method and  use it.

http://extensionmethod.net/

I hope you have a clear view of extension method now.

Enjoy!!

JavaScript can be easy with CoffeeScript a nice Idea

Recently, I got chance to view Coffee Script. If you are not aware what is Coffee script then don’t worry this post will help you to understand it.

 

I am pretty much sure once you work with this you will fall in love with this language.

 

So, without wasting time let me tell you what Coffeescript.org says

“CoffeeScript is a little language that compiles into JavaScript”

 

It means that it is a simple little language which convert the your simple coffee script statement in Java script. This conversion is seamless.  As I mentioned above Coffee Script is simple, easy to Understand , readable , maintainable and reliable.

 

Now, you are thinking it might be slow  so this is not the case. It is fast.

 

The only thing which we need to take care when we will working on this is White Space as it is very sensitive for white space. So , we need to do extra precautions when working on it.

 

I think we discuss lot of theory not it is time to do some practical.

 

Although, we can use Visual studio as well. In visual studio we have to add .Coffee extension’s file.

for this post demo I am using  Live coffee script IDE which is http://fiddlesalad.com/coffeescript/   

 

so, let’s open the Live IDE 

IDE1

 

Or we can use http://coffeescript.org/  “Try coffeescript” tab as well

 

IDE2

 

Now let’s understand it by few examples

So We don’t need to use semi colon “;” symbol for statement terminator CoffeeScript is using white-Space delimiter and also we don’t need to use curly braces “{}”  we can use indent or new line instead of “{}”.

Let’s understand this by an example suppose you want to write a JavaScript function with following features

1. Accept 2 parameter a & b

2. Add a &  b value and assign in a new variable which is C

3. Now add a condition If  C’s value is greater than 100 then show alert message “ more than 100”

4. if  C’s value is less than 100 then show alert message  “Less than 100”

5. If C’s value is equal to 100 then show alert message “ Equal to 100”

 

Now suppose if the above function we need to create in JavaScript then we need to write code as shown in below figure

example_JavaScript_1

 

if same thing you want to achieve in CoffeeScript then you need to write code as shown below. If you see there is no curly braces & semi colon.

CoffeeScript_Example_1

 

I am pretty much sure after seeing above example you might be excited to know more about it. 

To make you excited let write a simple CoffeeScript below and then we will compare it with JavaScript code which we usually write.

Here our objective is to create a Vehicle class with constructor and with Mileage

CoffeeScript2

 

Now to achieve same thing in JavaScript we need to write code as shown in below figure.

JavaScript_2

 

I am pretty much sure you are now super excited to know more about CoffeeScript.

I hope this tutorial might be helpful and interesting.

 

RJ

The Step by Step Azure Machine Learning is good Idea Part–3

In the previous post , we discussed about different algorithm types which help us in Machine Learning. Now, In this post, we will see how to work with Azure Machine learning studio.

So, let’s start with Step by Step how to use Microsoft Azure Machine learning.

Step1 :- You have to login into Microsoft Azure portal with your credential if you have not created yet then below is the link by which you can create your Azure account.

Step1
Step 2:- Click on Portal option of Machine learning as shown below. Here you have to click on “Create AN ML WORKSPACE” when you click on this option you will find the below the screen. You need to add workplace name and select the server location.

Step2
Once you done with above steps, you will find the below screen

Step3

Step 4:- You can open Azure ML studio by clicking Open Studio icon or you can click directly login into https://studio.azureml.net 

Once you successfully logged in you will get the following screen.

Step4

In this screen, you will see different options like existing experiments created by you, samples by Microsoft etc.

Step 5:- You can check existing samples as well which are on different data analysis algorithms.

step5

Step 6:-  But, let’s not go with existing sample as we will try to create something from scratch.  For that when you click on new Experiment then you get the following screen.

There is 3 main part of the studio screen. In Left panel, you will find different data sources , filters, transformation,models,Options. You just need to drag-drop on the central panel which will be our working area where as the right side panel will show the properties.

Blank_Experiment_Template

for our test, we can use  already uploaded dataset or upload new dataset on which we want to create the model. In this example, we are using Adult Census income which is already available in dataset repository of Azure Studio.

Although, we can download same data from http://archive.ics.uci.edu/ml/datasets/Census+Income as well  and upload to Azure Studio back. If we open the link you will find different attributes like  age, sex, marital status, race country  etc.

Before going further, let’s understand  what we are going to do?

So, we are going to  predict the income and determine/predict  whether the income is more than $50,000  or not. The training model  is based on the known inputs like age,education, job type, marital status, race and  number of hours worked per week.

Just drag drop the data source (dataset) on the central panel.

Choose_Data_Source

You can visualize the data by right click on the central circle of each control. You can check data anytime in any step.

DataVisualizationtion

Step 7:- Now, this dataset  might content some missing data values and for better result or prediction we will clean these data. To achieve this we need to drag-drop the Missing data control and provide dataset inputs. Here from the property panel you can set any value and replace it with the missing data value.

In this example, we will set the value to 0 (as shown in the figure). Now the output of this missing value is a proper dataset on which we can do the analysis.

Indiandotnet_CleanMissing_Data

step 8: Once we have proper dataset then our first goal is to split the data using split control . Here splitting data divide our dataset into 2 part

Indiandotnet_Split_Data_ML

1) Training data :- With the help of this data we will train the Machine

2) Validation data :- This data is used to check the accuracy of prediction which we will apply.

We can set the ratio of these two parts by the properties.

Step 9:- Here  we need to apply the algorithm. Assuming the prediction result set we are going to use Two-class classification model. If you see in the left panel there are many classifications. For this example Two-class Boosted decision tree.

Indiandotnet_two_class_Boosted_Decision

Step 10: Now, we have data with us on this data, we have to apply the algorithm and teach the model how to evaluate. for this, we will drag drop the “Train Model” control from left panel to our designer panel .

Indiandotnet_Train_Model

Here we need to configure “Train Model” . For this just provide the output of Split data (Training data output) to the Train model and also output of two class Boosted decision control’s output to train model. Once you set both the control then your screen will look like as below

Indiandotnet_TrainModel_1

Step 10:- Now once we have provided the input data to Train Model next thing is to  provide the name of the column which we have to predict. for this click on Train Model. Check the right panel and select the column which you need to predict  by clicking Launch column selector.

Indiandotnet_Select_Column

Step 11:- So far we are good, we have created the trained model now it’s time to score the model and cross-check the accuracy of the algorithm which we have applied. To evaluate the model we will drag drop a Score Model from the left panel and connect with Train model and the other validation data output of split data. Once you are done with this just press save button and run the model. if everything works well then there would be a right checkbox appear on control.

Indiandotnet_Score_Model

Step 12:- Once everything with green checkbox check as shown in the figure .

Indiandotnet_Saved_Score_Model

We will check data visualization by right clicking on score model.

Indiandotnet_Score_Model_Visualization

You will find 2 additional columns which are Scored Label & Score probabilities. These two columns provide scored value and probability percentage

Indiandotnet_Score_Card_Visualization2

Although, we can have different methods as well to achieve the same task and we can evaluate the result of these different methods by one more control which is Evaluate Module.  We have to drag-drop this module from left panel to designer and provide the Score model’s  output to the evaluate module.

Although, we have  only one training model with two class method but we can create another train model as well as compare them by providing output of the model to Evaluate module.

Indiandotnet_Evaluate_Model

It provides a set of curves and metrics which we need to understand and choose the best train model if we are evaluating two train model.

Indiandotnet_Graph

Till now we have created a model. here are not going to save the model as a trained model and publish it we try to expose this model using a web service.

Indiandotnet_Web_Service_Input

Step 13- Add output of web service dragging and dropping the web service output control and configure as shown below figure.

Indiandotnet_Web_Service_Input_Output_ML

Step  14:- In the next step, we will publish the web service with the below button of the designer when you click on publish web service you will get the following screen after completion of the step.

Indiandotnet_Default_Api_Deployed

Here just click on Test operation and you are good to go with Azure Machine learning experience.

I hope you will try yourself. Please , keep sharing your inputs.

Enjoy

RJ