MS SQL Server

Extract Data from SharePoint List using SSIS

Microsoft released a new task called “OData Source” which helps us to connect to online SharePoint and extract the lists.

Follow the below steps to configure and extract the data.

1. Make sure to use the latest version of SQL Server Data Tools to work with SQL Server integration Projects.
2. Use the Data Flow Task
3. You can see “OData Source” task. Drag and drop into the data flow canvas.
4. This OData Source task required a connection manager. Click New on the OData Connection Manager.
5. You need to pass the below information to successfully connect with SharePoint Online.
   1. Service document location – https://<yoursharepointsitename>/_vti_bin/listdata.svc
   2. Authentication Type: Microsoft Online Services
   3. UserName: Your email address to connect SharePoint Online
   4. Password: your password.
6. Test the connection.
7. On the OData Source Editor, Choose the SharePoint list on the collection section.
8. Check the columns and use any destination where you want to store the data.

Screenshots.

Extract Data from SharePoint excel using SSIS

There is no direct way of extracting the SharePoint Excel data to SQL Server using SSIS. However, we can process multiple steps to achieve the result.

This post explains that how we can download the excel files from SharePoint and store it in a local drive and then process the stored excel from SSIS.

To download the files easily, we can use the tool called “SSIS Integration Toolkit for Microsoft SharePoint”.

Follow the below steps.

1. Visit – https://www.kingswaysoft.com/products/ssis-integration-toolkit-for-microsoft-sharepoint/download
2. Download 64 bit of Toolkit
3. Install it on your system where you have installed the SQL Server Data Tools
4. Create a new SQL Server Integration Projects
5. Drag and Drop the data flow task into the control flow
6. You could see “SharePoint Source” and “SharePoint Destination” tools under Common folder.
7. Before Drag and drop the “SharePoint Source” task, you should create a SharePoint connection Manager.
8. Configure SharePoint Connection Manager for SharePoint on-premises or online.
9. Test the connection once.
10. Drag and drop the “SharePoint Source”. Point the created SharePoint connection and Choose the source list as “Documents” or the library where you have stored the excel files.
11. Configure the Download Destination.
12. Once excel files are downloaded then you can use that excel files to process further using excel source tasks.

Screenshots.

Python scatter plot in SQL Server 2017

As you know, Microsoft released the latest version of SQL Server which is SQL Server 2017. If you worked with SQL Server 2016 then you could realize that Microsoft SQL Server is not just a relational database anymore because it started to support big data and so many options to handle the non-relational data.
R Language is very popular to work with data science-related projects. It was integrated with SQL Server 2016 and we can run the R Scripts in SQL Server Management Studio itself.
It helps us to avoid the data movement between the relational database to R server and process. The same way, Microsoft now introduced Python integration with SQL Server. These 2 languages are coming from machine learning services in SQL Server.
To run Python scripts in SQL Server management studio, you need to enable the external script stored procedure.
Run the below command in your SSMS and see whether you are getting the “hello world” as an output.
In case, you are getting an error message then the configuration part was not properly done.
execute sp_execute_external_script
@language = N’Python’,
@script = N’
print(“hello world”)’
The below script is to generate the scatter plot.
DECLARE @Query nvarchar(max) = N’SELECT Year, Sales from [dbo].[SalesByYear]’

execute sp_execute_external_script

@language = N’Python’,

@script = N’

import matplotlib.pyplot as myplot

X = myplot.figure()

myplot.scatter(InputDataSet.Year,InputDataSet.Sales)

myplot.xlabel (“Year”)

myplot.ylabel (“Sales”)

myplot.title (“Sales by Year”)

myplot.savefig (“D:\Win – 8\myfig.png”)

‘,

@input_data_1 = @Query
Check the below step by step procedure to create a scatter plot using python.

Graph database in SQL Server 2017

As you know Graph Database is one of the latest features from SQL Server 2017.
Let us first understand the purpose of Graph database. We have a relational database which handles most of the scenarios but as we are started to handle big data and complex scenario our database also should be capable enough to handle those scenarios.
Yes. Graph database handles those complex scenarios easily which I have explained. As part of graph database, Microsoft team introduced two different tables.
1. Node
2. Edge
Check out the below explanation of those tables and how to work with graph database.
Use the below scripts

Use SQL2017
—Create Main table
CREATE TABLE People (
ID INT PRIMARY KEY,
Name NVARCHAR(25)
) AS NODE;

–Create Edge Table for relationships
CREATE TABLE RELATIONSHIP (
TYPE NVARCHAR(25)
) AS EDGE;

—Insert values to People
INSERT INTO People VALUES (1, ‘David’)
INSERT INTO People VALUES (2, ‘John’)

SELECT * FROM People;

SELECT * FROM RELATIONSHIP;

–Create relationships
INSERT INTO RELATIONSHIP VALUES (
(SELECT $node_id from People WHERE Name = ‘David’),
(SELECT $NODE_ID FROM People where Name = ‘John’), ‘Father’);

INSERT INTO RELATIONSHIP VALUES (
(SELECT $node_id from People WHERE Name = ‘John’),
(SELECT $NODE_ID FROM People where Name = ‘David’), ‘Son’);

—Cartesian Product result
–No need to use joins since nodes and edges are interconnected in structure
SELECT FromName.Name, RELATIONSHIP.TYPE, ToName.Name
FROM People AS FromName, People As ToName , RELATIONSHIP

–Proper Result
SELECT FromName.Name, RELATIONSHIP.TYPE, ToName.Name
FROM People AS FromName, People As ToName , RELATIONSHIP
WHERE MATCH (FromName-(Relationship)->ToName)

–more Records
INSERT INTO People VALUES (3, ‘Nancy’)

INSERT INTO RELATIONSHIP VALUES (
(SELECT $node_id from People WHERE Name = ‘Nancy’),
(SELECT $NODE_ID FROM People where Name = ‘David’), ‘Daughter’);

INSERT INTO RELATIONSHIP VALUES (
(SELECT $node_id from People WHERE Name = ‘David’),
(SELECT $NODE_ID FROM People where Name = ‘Nancy’), ‘Father’);

INSERT INTO RELATIONSHIP VALUES (
(SELECT $node_id from People WHERE Name = ‘John’),
(SELECT $NODE_ID FROM People where Name = ‘Nancy’), ‘Brother’);

INSERT INTO RELATIONSHIP VALUES (
(SELECT $node_id from People WHERE Name = ‘Nancy’),
(SELECT $NODE_ID FROM People where Name = ‘John’), ‘Sister’);

SELECT FromName.Name, RELATIONSHIP.TYPE, ToName.Name
FROM People AS FromName, People As ToName , RELATIONSHIP
WHERE MATCH (FromName-(Relationship)->ToName)

SELECT FromName.Name, RELATIONSHIP.TYPE, ToName.Name
FROM People AS FromName, People As ToName , RELATIONSHIP
WHERE MATCH (FromName-(Relationship)->ToName)
AND FromName.Name = ‘Nancy’

Share your comments below. Thank you

Windows could not start the SQL Server – Error & Solution

This post explains you very basic error that you get rarely and solution to that error,

Error: “Windows could not start the SQL Server on Local Computer. For more information, review the System Event log”.

The below error message will appear when your SQL Server evaluation period is expired.

Check the information in Event Viewer.

Solution

Buy the server and use the product key.

You can visit Maintenance in SQL Server installation and can upgrade.

Latest Update on SQL Server Management Studio

This simple post highlights the top level updates on components in SQL Server Management Studio.

The latest release of SQL Server Management Studio v17 RC3 has updates on below section compare than the earlier version.

1. Microsoft SQL Server Management Studio
2. Microsoft Analysis Services Client Tools

SQL Server Drop Database User

This blog post explains the issue that we usually face while dropping the users from the database and how can we resolve the issue.

We can create a user for database easily by any of the following user types,

1. SQL User with login
2. SQL User without login
3. User mapped to a certificate
4. User mapped to a asymmetric key
5. Windows User

If we want to drop the user then we can write drop statement but it will throw a below error in case the user owned any of the schemas.

Msg 15138, Level 16, State 1, Line 20

The database principal owns a schema in the database, and cannot be dropped.

To resolve the issue, Follow the below steps.

Step 1: Check the schema that owned by the user.

SELECT *

FROM sys.schemas s

WHERE s.principal_id = USER_ID(‘Username’)

Step 2: Change the schema back to dbo.

ALTER AUTHORIZATION ON SCHEMA::<schemaname> TO dbo;

Step 3: Drop the user

DROP USER <username>

SQL Query – Network Bandwidth Optimization

This article explains that how can we improve the performance when we access the SQL Query via network.

Scenario: I have taken SQL Server installed on Virtual Machine with 7GB of RAM and accessing the SQL Server database from local SQL Server management studio.

Step 1: Up and Run VM with SQL Server installed.

Step 2: Enable the DNS name of the Virtual Machine.

Step 3: Enable the TCP port with 1433 in portal.

Step 4: Create TCP port inside virtual Machine. Type WF.msc in command prompt. Right click on Inbound Rules and choose New.

Step 5: Make sure SQL Server services are up and running.

Step 6: Enable the SQL Server as mixed mode authentication.

Step 7: Create a new user login and check.

Step 8: Access the database from local SSMS.

Performance Improvement with NitroAccelerator

Step 1:

Install the “NitroAccelerator” in local machine, http://nitrosphere.com/trial/?ref=83095

Step 2: Run the query in local SSMS without Nitro Accelerator.

Step 3: Start the accelerator and test the query performance.