The Step by Step Azure Machine Learning is good Idea Part–2

In the last post, we have discussed Machine learning. Now in this post, we will discuss some more detail about algorithms and trust me this is one of the most important objects which a Machine learning engineer should know.

These objects are nothing but the algorithms. As a data scientist or Machine learning engineer, the first and most important thing are we should know

what is the data ?

What the result is ?

what analysis do you need to apply to get the desired result or prediction?

I know this is pretty much clear but let me explain with an example. Suppose, we have students data with high school ‘s internal assignments and we need to predict if the student can be pass in the final exam or not.

Now let me give you a brief overview of some of the algorithm  types which we may require in Azure Machine Learning. Although, there are much more types ,subtypes available but will not go in deep. So, let’s start

1) Two- Class  Algorithm Type:-

We will apply this algorithm type when the prediction result in either Yes/No or true/false or 1/0. for example, a student can be pass or not.

http://www.dreamstime.com/-image9248825

2) Classification Algorithm Type:-

This is another algorithm type which help us to predict  answer like which Kabaddi team or cricket you will cheer or which political team you will vote.

 

3) Linear Regression  Algorithm Type:-

This is one of the common prediction methods which everyone applies Smile sometimes. for example, in office, you can predict an engineer’s salary range depending upon last few engineer’s salary, prediction of property selling amount range Like this plot might be from 20 lac- 25 lac depending on last few years property price.

 

Indiandotnet_Predict

4) Anomaly detection  Algorithm Type:-

By the name, it is clear we need to find anomalies. for example, you have to determine from a group of  white cows and black cow you need to find out odd color cow means black color cow.

 

Indiandotnet_Anomoaly

 

I hope the all the above algorithm types is clear. In next post, we actually do the step by step Microsoft Azure Learning so don’t worry about that.

Please, provide your inputs

The Step by Step Azure Machine Learning is good Idea Part–1

Machine learning is not new in the market but nowadays  it is a buzz word everywhere. You might realize that there are lots of things happening in the Machine Learning.

Many big companies like Microsoft, Oracle, IBM,SAP and many other working in this area. They have provided Azure Machine Learning,Oracle Advanced Analytics, IBM SPS, SAP Predictive Analysis tools to work on it.

Before jumping into Azure Machine Learning directly let’s first understand basic of Machine Learning what exactly it is.

So, Machine learning is a way to understand the data pattern , recognize it and predict accordingly for future.  It helps in

1) Data Mining

2) Language Processing

3) Image recognition

and many other Artificial Intelligence  related stuff.

I know above statement is bit bookish so let me explain in Indiandotnet style .

Let’s say you are a teacher in  a school and you have quite experience in teaching. In each year you teach many students you also keep previous years data and some sort of basic detail of students.

When parent’s come to meet and wants to know the progress and whether he/she will pass in graduation or not. You simply do data analysis in your mind  whether that student doing study  or not (obvious), what his/her percentage in the last couple of exams or internal assessments , how he/she performed in previous class etc., then you give your prediction to the parents that their child does good or bad in the final exam or not.

Now, suppose instead of you there is a computer and parents asking the same question to the computer Now, a computer should provide the same answer as you give accurately or might be better.

For this, we need to feed enough data sample in the computer. If he has previous data samples by which he can analysis and predict accurately.

This overall exercise of processing data is part of Machine learning.

So, firstly you have to train the computer with providing the initial data which we can say training data. This is an iterative process.

 

Although, Machine learning Is more than this. Here, we are showing some more example where machine learning can help

1) Detecting  fraud credit card

2) Determine SPAM emails

3) Provide customer like to switch to competitor

4) Free text when typing etc. many more examples of machine learning.

There is 2 distinction in machine learning

 

1) Supervised Machine Learning :-

The Supervised learning means the value you want to predict is already exist in training data. Means the data already exist in the computer so data is labeled. The accuracy is high in such case.

 

2) Unsupervised machine Learning :-

So It is just opposite to Supervised Machine Learning. In this the predictive data not present in training data.

I hope now we have a basic understanding of Machine Learning. In next post, I will share step by step example of Azure machine learning.

 

Please, provide your inputs.

How Easy Automation testing of web application with Visual Studio – Part II

In the previous post, we saw how can we test a windows application using Coded UI. In this post, I am trying to describe how we can do automation testing of a web application .

So, let’s start step by step for

Automation testing of a web application.

 

Although, I am showing Automation testing in this post  but the project template  is more than that. I will describe in future posts for load testing & web performance related concept as well using this great tool but In this post, we are just concentrating automation testing of  a website.

 

Step1:- Open visual studio and add a new Project. This time, we are using “Web Performance and load test project”  as shown in below screen. Although, this template is self-descriptive.

 

1

 

Step 2: – When you click on this you will get the following screen.

2

 

Step 3:- Here, if you saw above image you will notice we have to click on recorder button. When you click this button, you will find internet explorer screen with record, pause & stop options as shown in below figure.

 

3

 

Step 4:- Here we have to provide the URL which we want to test. In this example, we are using http://nerdTechies.com and for a demo purpose, we are clicking on different menus and searching with a specific value. All our activities which we will perform will be recorded by the recorder and we can easily see those steps as well. As you can see in below figure.

 

4

 

Once we done with our basic test cases we have to press STOP button as shown in the above image. when we click STOP button our Internet explorer browser will be close and we will get all the recorded action in our Visual Studio IDE as shown in below image.

5

 

Step 5:- Now, in the next step we defiantly want to cross check all the action which we performed and recorded by the recorder in Internet Explorer. for this, we  just need to click on Run Test button  as shown in below figure.

6

 

Step 6:- When we Click run test option the IDE run all the recorded steps in the same sequence and if everything as per the expected result it will display “Passed” status for that particular recorded test case as shown in below figure.

7

 

I hope you like this simple way of recording your test steps and cross check it again and again by just pressing Run test button.

 

Although, Just wanted to share as mentioned in the introduction of this post as well. This project can do much more this but for now , I am showing a small part of it.

 

I hope you will use it soon. I appreciate your inputs for this post.

 

Enjoy !!

RJ !!!

How to make UI Testing Easy with CODED UI ( A Glimpse)

In the last post, we discussed how to be a smart developer in testing. To continue that smartness Smile I am sharing one of the best tools which are CODED UI. Smile

 

A CODED UI is again a record and play concept what I mean here that you can record steps of your test case and play again and again whenever you want.

 

I know I made a simple statement for CODED UI but it is much more than that and you have full control on it. You can write your own custom code in CODED UI.

 

Although, this post is just starting or a basic information of CODED UI but it has many more things which we will cover later in our posts.

 

To understand, It better let’s start a step by step process. In this step by step process, I am using Visual Studio 2015. 

 

The best part of CODED UI is that you can record test cases of windows & the web both the application.

 

Here we are taking the simple example of UI recording which is open a Calculator and perform the operation and another one is the open website in the browser and record the steps (in Next post).

 

Step 1:- Open Visual Studio 2015. Create New Project and select CODED UI project as shown in below image. Here if you there are other options as well like Unit Test Project, Web Performance, and Load Testing Project.

So, those have their own unique properties which we will discuss soon.

Indiandotnet_CodedUI_Step_1

 

Step 2:- When you click the OK button in above window you will get project is opened in Visual studio with certain predefine features and a new pop option appeared as shown in below figure.

It has 2 options which are self-explanatory. We are going with default option which is “Record action, edit UI map or add assertions.

Indiandotnet_CodedUI_Step_2

 

Step 3:-  When you click this option then on your machine near task bar you will find UIMAP Windows. Which is responsible for recording your action.

Indiandotnet_CodedUI_Step_3

 

Step 4:- You can add Coded UI Test file by right clicking the project or add new Item option. This file contains all the recording steps. If the UImap is already open then you have to close it first and then add Coded UI test file.

Indiandotnet_Coded_UI_Step4

 

Step 5: Once you added the file and UImap will appear then next step is start recording your steps of test cases. Here we are going to open the calculator and adding 2 numbers. so firstly click on the Record button of UIMap so ,it will start recording the screen. If you see below screen you will find the recording button change to the pause button.

Indiandotet_UI_Step5

 

Step 6:- Now we clicked on calculator button. It appears in front of us. We added first value 10 and then second value 20 and clicked on an equal button. then press pause button (as our first test case is completed). As the best practices we have to go step by step and record it but for this example, I am not following that. If you click the button nearby pause button as shown in below figure you will get all the recorded actions so far

 

Indiandotnet_Coded_UI_Step_6

 

Step 7:- Once you confirm with your Recording then you can click on Generate Code button and if you have any doubt then you can re-record or delete that particular recording. If you observe above steps I deleted Click  ‘1’ button explicitly. I have recreated recording then clicked on Generated Code button. When I clicked I got the following option. Here you need to Provide method name and then save it.

Indiandotnet_Coded_UI_Step_7

 

Step 8:- Now to check whether the added value result is correct or not whether we have to fail this test case or pass we have to click on the guy as shown in below figure

Indiandotnet_CodedUI_8

 

Now, this is bit tricky part you have drag cross arrow from this control to Calculator’s result window and highlight as show in the below figure.

Indiandotnet_Coded_UI_10

 

Once this done then add Add Assertion by clicking the Add Assertion option as shown above image. When you click you will get the following screen

Indiandotnet_Coded_UI_9

 

Once the above step is created Just click on Generate code once again. So that it will add Assertion logic as well.

So , far we have created a Coded UI test cases. Now , the challenge is how to cross check it. So here are  steps.

Click on Test Option from Visual Studio menu  click on Run option then click on  All Test case options.
You will find all the steps that you perform (recorded) earlier will re-run and after execution, you can cross check. The test explorer.

If all the steps of test cases pass then according to assertion logic our test case will pass. as shown in below figure.

Indiandotnet_Coded_UI_12

 

In this way, We have captured a windows application test cases and record and played accordingly. In the next step, I will show you how easy to cross check web application as well.

Please, provide your input what you think of this article.

 

Happy weekend!

Enjoy.

Finally SQL Server 2016 Released on 1st June 2016

Yes, you saw correct news. SQL SERVER 2016 finally released on 1st June 2016. It is most awaited NEWS. You will find many surprising features in this Build.

Below are some upcoming features which you will see in this feature

  1. Temporal Table
  2. Stretch Database
  3. Row Level Security
  4. Poly Base
  5. Running Estimation Plan
  6. JSON Support
  7. Security Data Masking
  8. Always Encrypted

SQL_SERVER_2016_

And many more. We will discuss it soon on our blog

Mean while you can download SQL SERVER 2016 from link Download.

And to read more about this release you can check at  SQL Blog

Till than Enjoy.

How to automate web UI testing for smart developers ?

As a developer, I do agree that we have to do Unit testing of the code  and most of us write different unit test program and execute them but this is not enough at our end. It is a developer responsibility to test the application at his/her hand before deploying into the QA environment.

Now, It feels sometime boring to test same page again and again or check whole flow of application every time indeed it is time consuming as well. Here. In this series, We are going to see various different options by which we can ensure that everything which wrote earlier or the  previous functionalities are also working fine. As a developer this is our first and for most important to cross check our work and also make sure everything is working fine after our changes.

Here in this post I am sharing one simple smart step to test your website again and again by just one click. This requires only first time efforts.

So, the magical thing is “Selenium IDE“ which is a FireFox  Add-on. You can download and install from below URL

https://addons.mozilla.org/en-US/firefox/addon/selenium-ide/

Selenium_IDE

Once, you installed you will get the icon as shown below.

installedIDE

Now, when you click on this Selenium icon you will get following screen.

seleinum_ide_Indiandotnet_1

To make it simple to understand, we are using a simple example. Here in Base URL we are adding  http://NerdTechies.com  and clicking on Record button as shown in below figure

seleinum_ide_Indiandotnet_2

Once, you added base URL it means you are going to capture all the happening on this base URL.

Now open this URL in Firefox browser and once the site is open click on different links which you want to test whether that working or not? The selenium IDE capture all the events which you performing. Once your think you tested all the mandatory things again click on the record button to stop the test case.

Now once your capture everything you can save this test case.

seleinum_ide_Indiandotnet_3

Once you saved it. You can run this test case later on as well. Now, Here you are thinking that you have capture the test cases , saved test cases but, how to run the test case.

seleinum_ide_Indiandotnet_4

When you run test case by clicking the play button as shown above figure you will see that , whatever action you performed earlier will repeat again in browser. If everything worked according to your last action then test case is successful else test case will fail.

Now, This is the simplest way to cross check your monotonous work (test cases) again and again by just one click.

In near future we will share two more important tips

1) Selenium with .NET

2) Coded UI  a .NET project

I hope meanwhile you will enjoy this selenium IDE with Firefox. I am pretty much sure once you start using it you will save your time and do smart work.

Enjoy !!

RJ!!

Why Do People Think Learn Step by Step PIVOT Transformation is a Good Idea ? TIP #125

In the series of step by step SSIS learning, this is another interesting article. In this article we will try to implement PIVOT transformation in few easy steps.

Let’s understand the PIVOT by below example. Suppose, you have a salary data of employee as shown below.

Employee Name Salary($) Year
Sandeep 110000 2016
Sunil 120000 2016
Shreya 130000 2015
Virendra 140000 2015
Sandeep 120000 2014
Sunil 130000 2014

Now, if our requirement is something like below table then we would require PIVOT functionality.

Employee Name 2016 2015 2014
Sandeep 110000 0 120000
Sunil 120000 0 130000
Shreya 0 130000 0
Virendra 0 140000 0

In general, we say when we want to convert Rows data to Columns then this is called Pivoting.

Now, lets create a sample of PIVOT transformation with few basic steps. In this example, we are again using AdventureWorks database. EmployeePayHistory, Employee, People tables are base of our example. We wrote query which fetch data similar as shown above.

4

Let’s move step by step

Step 1: Add a dtsx package and drag drop data flow task as show in below figure.

1

Step 2:- Now, double click on Data flow task and drag drop data source and select SQL Data source and configure the SQL statement as shown below.

2

3

Step 3:- Now, We have to drag drop PIVOT transformation control as shown below.

5

Step 4:- Now, we have to configure the  PIVOT control. When you choose edit option then you will get the following screen.

Here, we have to choose Pivot Key, Pivot Value and Set Key.  In the Pivot Value we use the value which we have to show in the columns.

 

15

If you see pivoted output columns highlighted in yellow in above image is the year wise columns. if you are sure for the columns which you want as a pivot columns then you can add the name of the columns in place of [value1],[value2] etc. and click on Generate Columns Now button. (In current case we have limited years so we have added [1997] to [2004] values in this textbox and clicked on Generated Columns Now button due to which we got yellow required Pivot output columns as highlighted yellow.

Step 5:- Our configuration is not yet complete to configure this we have to click on advance setting and select Input and output properties.

Here, if you want to add any extra column as an output column which exist in Input columns  then you have to select Pivot Default Output and click on add column. Now, the main important point if we want to map it with existing input source column then in such case we have to add LineageId of that particular column.(AS highlighted in below images)

7

8

Step 6:- The another most important point is PivotUsage which is highlighted below. If we want to add columns which are not affecting the pivot then we have to update the value of PivotUsage to 0.

9

Step 7:- Once the Pivot is configure the next step is to provide the output of pivot to a destination. As a destination we are using Excel file as usual Smile.

So, drag drop excel destination source and configure it as shown in below figure .I hope you will not face any issue in this. If you are new and facing any difficulties then please visit previous articles of Zero to Hero in SSIS Series

10

11

Step 8 :- Once the configuration is done we will see below screen.

12

Step 9:- Now, once everything is completed then we run the package and we will get following screen.

13

Now to cross check we open the excel and cross check the data. When we open the excel, we found desire result.

14 

So, We have achieved what we want. We did our first Pivot Transformation example.

Please, feel to drop your comment.

Enjoy !!!

RJ!!!

The Unconventional Guide to How to Implement Lookup Transformations TIP #124

This is one the articles in the series of step by step SSIS tutorial. In this post, we are going to understand Lookup transformation and also see an example to implement Lookup transformation in 8 easy steps.

Now, Lookup transformation is one of the interesting transformation which is used to implement equijoin between data sources.
We are discussing the same example which we discuss in the earlier post which is Person & PersonPone tables of AdventureWorks.

Suppose, we want Person details and the phone number of person as a result then we can apply Lookup transformation.

When, we applied the Lookup transformation between Person & Person Phone table then we will get two result set.

One of the result sets is the Matched result set in which we will get the Person data and the First Matched Phone number while the PersonPhone table might have multiple phones for a person but we will get only one phone number record because Lookup transformation uses EquiJoin.

In the second result set, we will get unmatched Person records which mean the Person who doesn’t has a Phone number in PersonPhone table.

I think, we have a basic understanding of what is Lookup Transformation. Now, we are going to implement Lookup Transformation step by step.

Step 1:-
So, the first basic step is to add a package and drag-drop data flow task as shown in below figure. You can rename the  data flow task  as I did in below figure.

1_DataSource

Step 2: – In this step drag drop a data source and configure it for Person table as shown in below figure. If you face any difficulties in this then you can review previous articles of this series

2_SqlSource

Step 3:- Once the Person table is configured just drag drop Lookup Transformation control and provide the input of Person table Data source to Lookup table. When you try to configure the Lookup you will get the following screen. We will discuss Cache mode separately in detail. In this example, we are using Full Cache and connection type to OLEDB connection Manager.

Now, Click on Connection option on the right and configure the PersonPhone table. 
4_Lookup

5_LookupTransformationColumn

Step 4:- Now, configure the Columns relationship between Person & PersonPhone table.To achieve this just drag-drop BusinessEntityId of available input columns of Person table to available Lookup columns which are in PersonPhone table and matched with BusinessEntityId which is common between Person & PersonPhone table. Here we have to select PhoneNumber column which we have to include in the result set.

4_Lookup_111

Step 5:- Now, Drag drop two excel destination source and configure it. As, I did in below figures. Here one Excel is with name LookupOutput.xls which contain matched records.

7_ExcelOutput

Below, is another Excel which contains unmatched records. Here UnMatched means the Person records who don’t have phone numbers.

8_Lookupoutputnotmatched

Step 6:- Now, If you see there are 3 output lines coming from Lookup transformation.So, First blue output gives it to Lookupoutput excel which will have all the matched records of person & Person Phone number. You can configure inputs as shown below.

9_Matched_Excel

Step 7:- In a similar way, we have to configure unmatched record output. Once, we done with configuration of both Matched & unmatched output, we will get following structure as shown in below screen

10_NotMatched_Excel

10_Final

Step 8:-  Once, everything is configured you can run the package and if everything is perfect then you will get the following screen

finalRun

I hope above steps are easy and you can implement Lookup Transformation at your end too.

Please, provide your inputs.

Enjoy !!!

RJ!!!