python

ByHariharan Rajendran

Python Script to Event Hub – Power BI Sentiment Analysis ML Function

This article explains the python script to send the data to event hub.

As we know, event hub is azure service which is designed to capture the streaming events from different devices or applications.

The below python script sends the sentiment string to event hub.

import sys

import logging

import datetime

import time

import os

import itertools

from azure.eventhub import EventHubClient, Sender, EventData

 

logger = logging.getLogger(“azure”)

 

ADDRESS = “amqps://<eventhubworkspacename>.servicebus.windows.net/<eventhubname>”

 

# SAS policy and key are not required if they are encoded in the URL

USER = “RootManageSharedAccessKey”

KEY = “<Key>”

 

try:

    if not ADDRESS:

        raise ValueError(“No EventHubs URL supplied.”)

 

    # Create Event Hubs client

    client = EventHubClient(ADDRESS, debug=False, username=USER, password=KEY)

    sender = client.add_sender(partition=”0″)

    client.run()

   

    try:

        start_time = time.time()

        for i in range(1):

            print(“Sending message: {}”.format(i))

            a=”[{“+”ts”+”:”+str(1550321522)+”,”+”Message”+”:”+”\”good\””+”,”+”TestVal”+”:”+str(7)+”}]”

           

       

            print(a)

            sender.send(EventData((a)))

                     

           

    except:

        raise

    finally:

        end_time = time.time()

        client.stop()

        run_time = end_time – start_time

        logger.info(“Runtime: {} seconds”.format(run_time))

 

except KeyboardInterrupt:

    pass

 

 

Applications

The above script sends the data to event hub which can use used as an input to the AZ ML function in Stream Analytics.

The output of the stream analytics query can be used for Power BI streaming dataset.

 

ByHariharan Rajendran

PowerBI – Python Support

Microsoft introduced Python support from Power BI on August 2018 update.

PowerBI is supporting R visuals and PowerBI producing custom R visuals too. PowerBI team extended the data science capability of PowerBI by supporting python.

In this post, I am explaining the simple python plot in Power BI.

Download the latest version of Power BI. As python support in preview support, we need to enable the python support preview feature.

 

You should install the Python on your machine so that you can point the python home directory which is mandatory to run any python scripts inside PowerBI.

Once you have configured the setup then you need to test with simple code. Use the below code.

import matplotlib

import numpy as np

import matplotlib.pyplot as plt

x = np.linspace(0, 3*4, 500)

plt.plot(x, np.sin(x**2))

plt.title(‘Test Plot’)

plt.show()

 

The above code works perfectly and produced the plot on jupyter notebook and need to get the same result on PowerBI also.

Drag and drop the python visual into the canvas area. In the below script editor, reference any dummy column and then copy and paste the above script.

Finally, execute the script and it will produce the plot as like below.

 

Now, you can play with your python scripts.

ByHariharan Rajendran

Python scatter plot in SQL Server 2017

As you know, Microsoft released the latest version of SQL Server which is SQL Server 2017. If you worked with SQL Server 2016 then you could realize that Microsoft SQL Server is not just a relational database anymore because it started to support big data and so many options to handle the non-relational data.
R Language is very popular to work with data science-related projects. It was integrated with SQL Server 2016 and we can run the R Scripts in SQL Server Management Studio itself.
It helps us to avoid the data movement between the relational database to R server and process. The same way, Microsoft now introduced Python integration with SQL Server. These 2 languages are coming from machine learning services in SQL Server.
To run Python scripts in SQL Server management studio, you need to enable the external script stored procedure.
Run the below command in your SSMS and see whether you are getting the “hello world” as an output.
In case, you are getting an error message then the configuration part was not properly done.
execute sp_execute_external_script
@language = N’Python’,
@script = N’
print(“hello world”)’
The below script is to generate the scatter plot.
DECLARE @Query nvarchar(max) = N’SELECT Year, Sales from [dbo].[SalesByYear]’

execute sp_execute_external_script

@language = N’Python’,

@script = N’

import matplotlib.pyplot as myplot

X = myplot.figure()

myplot.scatter(InputDataSet.Year,InputDataSet.Sales)

myplot.xlabel (“Year”)

myplot.ylabel (“Sales”)

myplot.title (“Sales by Year”)

myplot.savefig (“D:\Win – 8\myfig.png”)

‘,

@input_data_1 = @Query
Check the below step by step procedure to create a scatter plot using python.

1