Microsoft

Bing Image Search API – Python Script

This article explains how to use a python script to interact with Azure cognitive services – Bing Image Search.

Requirements

1. Python 3 & libraries
2. Bing Search API key – Azure Subscription

Bing Search API

Follow the below steps to create Bing Search API.

Step 1: Visit portal.azure.com

Step 2: Create a new resource – search for “Bing Search v7”

Step 3: Once created the API, you need to get API key. The base URL is same for everyone which is https://api.cognitive.microsoft.com/bing/v7.0/images/search

 

Python script

Use the below python script and replace the key.

import http.client, urllib.request, urllib.parse, urllib.error, base64

import requests, json

desc_url = ‘https://api.cognitive.microsoft.com/bing/v7.0/images/search’

headers = {

    # Request headers

    ‘Ocp-Apim-Subscription-Key’: ‘795bff7c586c43749d79a746dcebe12b’,

}

params = {

    # Request parameters

    ‘q’: ‘Hariharan Rajendran’,

    ‘count’: ‘5’,

    ‘offset’: ‘0’,

    ‘mkt’: ‘en-us’,

    ‘safeSearch’: ‘Moderate’,

}

desc_response = requests.get(

              desc_url, headers=headers, params=params)

desc=json.loads(desc_response.content)

print(desc)

You can change the parameter values to get the different result.

 

Power BI Report using SSRS Shared Datasets

Creating a Power BI report is very easy, and it supports loads of data connectors. When we start creating a report against any relational database, we must choose any of the below query modes,
1. Import
2. Direct Query or Connect Live

For example, when you try to create a report against SQL Server database, then you need to choose any of the above methods based on the requirement.

If you take SSRS report server, we have below elements available
1. Paginated Reports
2. Mobile Reports
3. KPIs
4. Data Sources
5. Datasets

If you are SSRS administrator, then you can easily manage all these stuff in a centralized place.

Is it possible to use the SSRS dataset to create a Power BI Report?
Yes, it is possible. You must have the latest version of Power BI Report Server.
We have REST APIs and is supported in Power BI Report Server, which is an extended version of SSRS 2017. Check REST API on SSRS 2017.

We need to use the OData Feed data connector to connect with SSRS datasets.
Below is the information required to create a report.
1. REST API Url
2. REST API Url with specific dataset id

The base URL of the REST API will be like below,
http:///Reports/api/v2.0

Avoid Using Dataset Filters in SSRS

If you are Microsoft SQL Server Report Developer, then you have a chance to notice the dataset properties as it is the place where we supply the data subsets to the report.

Let us understand the filters in SSRS first and analyse why dataset filters should be avoided.

We have three different main places to filter the data in SSRS

1. T-SQL Script
2. Dataset Filters
3. Run-time Filter using Parameters

Filter in T-SQL Script

When we understand the business requirements, need to categories what are the filters static and dynamic.

Always use the static filters in T-SQL script. Dynamic filters can be done through parameters but still need to add the filter in T-SQL Script with parameters.

For example,

If you want to filter country values as the US on your country table.

SELECT State, City, Country FROM dbo.COUNTRY

WHERE Country=’US’

Note: In this case, we are restricting data in database server itself.

Dataset Filters

You can filter in SSRS report dataset.

Go to Dataset -> Filters

Note: In this case, we are taking all the data to report server and filtering the data in the report.

Run-time filter using Parameter

It is a special kind of filter because this will be the requirement from business in most of the cases.

First 2 filters won’t be visible to end users but this filter will be visible and user needs to filter the value while running the report.

This can be achieved by adding the parameter in T-SQL script,

SELECT State, City, Country FROM dbo.COUNTRY

WHERE Country=@Country

Why should we avoid dataset filters?

The main reason is to improve the performance of the report.

In case 1 (Filter in  T-SQL Script), we are filtering the data in database layer itself. It means we are taking only required data to the reporting layer and showing the result. The performance of the report will be good as it is processing only fewer records.

In case 2 (Dataset filter), we are taking all the records from table database and filtering in the report server. It means report server has to make process and filter the data and show the result. We are giving burden to report server so it will affect the performance.

 

T-SQL Basics – Day 1

If you are reading this article then you are looking an very easy way to understand the T-SQL Scripts.

Let us learn T-SQL with a scenario, consider you have a sheet with some employee information. Your boss asking questions from that sheet and you need to answer him. In simple, your task is “Read from Sheet”.

Convert the general words into technical terms,

Sheet – Table (Employee) in a Database

Questions – T-SQL Statement

Answer – Output of T-SQL

Read – Select

Read from Sheet – Select from Table

If you are familiar with above terms then you can easily learn T-SQL Statements.

Let’s start,

Boss: What are the information available of an employee?

You: Read all the field name from Sheet

T-SQL: SELECT * FROM Employee

Note: “*” denotes all the fields from a table.

Boss: Do we have employee Name, Age & Country?

You: Yes

T-SQL: SELECT Name, Age, Country FROM Employee

Boss: What is the age of Richard? Or what is the country name of Richard?

You: 42 or UK

T-SQL: SELECT * FROM Employee Where Name=’Richard’

Note: Need to use WHERE keyword when we say specific data.

Boss: Who are all crossed age 40?

You: Read from sheet

T-SQL: SELECT * FROM Employee WHERE Age > 40

Note: Need to use WHERE keyword when we say specific data.

Boss: Whose names start with letter K?

You: Read from sheet

T-SQL: SELECT * FROM Employee WHERE Name LIKE ‘K_%’

Note: Need to use WHERE keyword when we say specific data. Use LIKE keyword when you want to play with the letters or pattern in a value. There are few other patterns available for LIKE which will discuss in the next article.

Boss: Convert the values of a Marital Status column into detailed values.

You: Add a new column

T-SQL: SELECT *, CASE WHEN [Marital Status] =’M’ THEN ‘Married’ ELSE ‘UnMarried’ End MStatus FROM Employee  

Note: Use “CASE WHEN XXXX THEN XXXXX ELSE XXXXX END” Syntax when you want derive something. We can discuss the detailed version above syntax in next article.

 

Custom Report for SSMS

It is very easy to create a custom report and use it inside SQL Server Management Studio.

As we know, we have standard reports from Microsoft to monitor the Server and Database information.

These reports are available for instance and database level, If you are not aware of this standard reports, please check it below to access those reports.

 

Steps to use custom report in SQL Server Management Studio.

Just created a Report as like below,

We need to adjust the data source properties to use inside SSMS. Modify the data source like below. Once modified the data source, save the report. We will take this rdl file and attach in SSMS.

Once added the report, open and see the information.

Now, you can refresh the report and see the real time information and even you can build a report to specific database for monitoring purpose.

Happy Learning.

Always Encrypted–PowerShell Scripts

We have discussed that how to create a Always encrypted columns and steps to create custom column master key & column encryption key.

This article explains about the creation of keys with the help of PowerShell scripts.

Latest PowerShell Cmdelt’s update from Microsoft, https://blogs.technet.microsoft.com/dataplatforminsider/2016/06/30/sql-powershell-july-2016-update/

Pre-Requisites:

Since Always Encrypted feature introduced in SQL Server 2016, we need to upgrade our SQLPS.exe. If you use the older version of SQLPS then that will not recognize the some of the below cmdlets. So make sure to upgrade SQLPS.exe first.

You can upgrade by installing the latest version of SSMS (SQL Server management studio) 2016.

Check for the below modules installed in PowerShell,

#List directory wise installed modules

Get-Module –ListAvailable

Check for SQLASCMDLETS & SQLPS. If there is no modules then make sure you have SQLPS.exe file available in below location

C:\Program Files (x86)\Microsoft SQL Server\130\Tools\PowerShell\Modules

# Import the SQL Server Module.
Import-Module Sqlps -DisableNameChecking;

 

Steps to Create a Column Master Key & Column Encryption Key.

In the Below script, we need to specify following information,

• Server Name
• Database Name
• Column Master Key Name
• Column Encryption Key Name

First Connect to the Server and Database using PowerShell Script

# Connect to your database.
$serverName = “<server name>”
$databaseName = “<database name>”

$connStr = “Server = ” + $serverName + “; Database = ” + $databaseName + “; Integrated Security = True”
$connection = New-Object Microsoft.SqlServer.Management.Common.ServerConnection
$connection.ConnectionString = $connStr
$connection.Connect()
$server = New-Object Microsoft.SqlServer.Management.Smo.Server($connection)
$database = $server.Databases[$databaseName]

As we know, we have different key stores available. This below script using “Windows Certification Store – Current User”.

# Create a column master key in Windows Certificate Store.
$cert1 = New-SelfSignedCertificate -Subject “AlwaysEncryptedCert” -CertStoreLocation Cert:CurrentUser\My -KeyExportPolicy Exportable -Type DocumentEncryptionCert -KeyUsage DataEncipherment -KeySpec KeyExchange

In the above script, you can check the “CertStoreLocation”. To identify the location, you can check in T-SQL Statement to create master key. I just highlighted in below script for understanding purpose.

USE [Sample_Encrypt]
CREATE COLUMN MASTER KEY [test1]
WITH
(
    KEY_STORE_PROVIDER_NAME = N’MSSQL_CERTIFICATE_STORE’,
    KEY_PATH = N’CurrentUser/My/55A4DDAF58D274547B9C3BA5ABA154662CDF62D0′
)

GO

# Create a SqlColumnMasterKeySettings object for your column master key.
$cmkSettings = New-SqlCertificateStoreColumnMasterKeySettings -CertificateStoreLocation “CurrentUser” -Thumbprint $cert1.Thumbprint

# Create column master key metadata in the database.
$cmkName = “<cmk name>”
New-SqlColumnMasterKey -Name $cmkName -InputObject $database -ColumnMasterKeySettings $cmkSettings

# Generate a column encryption key, encrypt it with the column master key and create column encryption key metadata in the database.
$cekName = “<cek name>”
New-SqlColumnEncryptionKey -Name $cekName -InputObject $database -ColumnMasterKey $cmkName

Verify the keys in SQL Server Database Security folder.

Always Encrypted–Key Stores

As you know, we can create “Column Master Key” & “Column Encryption Key” to encrypt the column values. You can check here to learn about key creation. Usually these keys are associated with the certificate which includes the details like,

Issued To, Issued By, Expiration Date, Thumbprint & etc.

In Always encryption, we have four different Key store options are available which helps us to secure our keys. These keys are associated with the table columns. We can decrypt the values using these keys so we need to make sure to secure the keys.

Key stores,

1. Windows Certificate Store – Current User
2. Windows Certificate Store – Local Machine
3. Azure Key Vault
4. Key Storage Provider (CNG)

 

Widows Certificate Store – Current User

This type of certificate store is local to current user on the computer. This is located in the registry of the computer, specifically on HKEY_CURRENT_USER root

We need to compare the current user certificate store with local machine certificate store. Current user certificate always inherit the content of the local machine certificate.

We can generate the certificate by our self and that will be available for one year and we can use that to encrypt the column values.

T-SQL Statement to create a master key, we can select the certificate (Thumbprint) which I highlighted.

Windows Certificate Store – Local Machine

This will be available on the computer which is global to all users on the computer. This is located under HKEY_LOCAL_MACHINE registry.

This will list out all the available certificates in your computer and choose for encryption.

 

Azure Key Vault

This should be used to safeguard the keys in cloud (Azure). It requires a Azure subscription.

 

Key Storage Provider (CNG)

It provides different providers,

1. Microsoft Passport Key Storage Provider
2. Microsoft Smart Card Key Storage Provider
3. Microsoft Software Key Storage Provider

We can choose any of the above provider, for more information about the algorithm, key length & etc., check Microsoft article here.

Always Encrypted in SQL Server – Part 2

We can easily enable the encryption using wizard which you can check it here.

Disabling the encryption is also an easy task through wizard.

Steps to disable the encryption,

Step 1: Right Click on your database and choose Tasks->Encrypt Columns..

Step 2: Select next on the introduction page.

Step 3: When you are enter into column selection page, you can see the lock symbol on the columns which you enabled the encryption. Under encryption type, you can see the drop down values which includes “Plaintext”, “Deterministic” & “Randomized”. Choose the “Plaintext” and proceed next.

Step 4: Go through further steps and finish it.

Step 5: Now, run the select statement on table and you should see the result without an encryption.

 

We can define the “Encryption Key” & “Master key” Name manually.

Follow the steps to create custom names and to make use for encryption.

Step 1: Go to Security folder of your database.

Step 2: Expand the security folder and you should see the folder called “Always Encrypted Keys”. Expand the folder.

Step 3: You should see another two folders, Column Master Keys & Column Encryption Keys. Right click on Column Master Keys folder and choose “New Column Master Key..”

Step 4: Define the name of the master key and choose the key store where you want to store your key safely and click OK. Master will be created.

Step 5: Right click on Column Encryption Keys folder and choose “New Column Encryption Keys”

Step 6: Give the name and choose the master key from drop down. You should see the master key which you created in above step 4 and click OK.

Step 7: Now, go and create an encrypted column and you should see the option to choose your encryption key. Master key also will be applied automatically.