AI Workbench

AI Workbench is a combination of ML Studio, previously known as Flow Studio, Notebook, Tasks and Flows.

ML Studio

Machine Learning Studio is a cloud-based, integrated development environment (IDE) for building, deploying and managing machine learning models. It provides a visual interface for building models and allows users to drag and drop modules to create workflows, without requiring extensive programming knowledge. The platform offers a wide range of algorithms and tools for data preparation, model training and deployment, making it easy to build and deploy machine learning models.

ML Studio also provides a rich library of pre-built, reusable modules, as well as the ability to build and share custom modules. This makes it possible to collaborate and scale machine learning projects, whether you are working alone or with a team.

ML Studio provides a cloud-based environment, meaning you can access your models and data from anywhere with an internet connection. This makes it easy to work on projects and share your work with others.

ML Studio was earlier called as Flow Studio. You can now create a flow without using Jupyter Notebook. ML Studio focuses on the creation of flow and functions associated with it. So, just as we can create a flow and perform various actions on it in Notebook, we can do the same in ML Studio.

ML Studio

By navigating to ML Studio, you can create a new flow by clicking Create New on the right side of your ML Studio window.

Creating ML Studio

While creating a flow, you can enter your flow name and give an description for that flow. After filling the details, you can click on Create and your flow is now created.

Creating ML Studio

Once you proceed with create, your screen will navigate to the below page. Here you can add the required number of parameters, select the required task and provide the values for your parameters.

Flow Creation - ML Studio

After providing the values, you can save your flow to your flowLibrary and you can run, register and promote your flow here itself.

Flow Creation - ML Studio

Just by doing drag and drop action, you can create your own flow as per your requirement.

Notebook

Jupyter Notebook is a powerful, open-source web application used extensively in data science, scientific computing and machine learning. It enables users to create and share dynamic documents containing live code, equations, visualizations and narrative text. The interactive nature of Jupyter notebooks enables users to run code in real-time and visualize the output, making it easy to experiment and iterate on ideas. Additionally, the ability to save and share notebooks allows for seamless collaboration and reproducibility. Supporting multiple programming languages, including Python, R, Julia and more, Jupyter Notebook is a versatile tool for data analysis and research.

The Notebook acts as the cornerstone of the AI Workbench, laying the groundwork for all other components to build upon. By guiding users to the Jupyter Notebook server, it grants access to a dynamic and immersive online hub for exploring data science and machine learning. With the Notebook in place, running resource-intensive code becomes a breeze, thanks to the already established platform.

With the ConverSight Library and CsSDK Library, the Jupyter Notebook becomes an even more powerful tool for data scientists and machine learning engineers. The ConverSight Library provides a wide range of pre-built tasks and flows, making it easier for users to accomplish complex tasks without having to write code from scratch.The ConverSight Library provides data processing functionalities for the user. What’s even better is that the ConverSight Library allows users to run these tasks and flows in the ConverSight Cloud, which is a cost-efficient option for those who don’t have the resources to set up their own servers.

Meanwhile, the CsSDK Library is an optimized and updated version of ConverSight Library. The CsSDK Library allows users to connect to existing datasets and configure them in more secured manner. This feature saves time and effort by allowing users to reuse templates and automate the data refreshing process through scheduled republishing. With these capabilities, the Jupyter Notebook and its associated libraries become a complete and robust solution for data analysis and machine learning tasks.

Accessing a Noteboook

To access Notebook in the ConverSight platform, navigate to the Configuration section and choose Notebook under AI Workbench.

Accessing Notebook

ConverSight provides two default Base Notebooks: the Standard ConverSight Notebook and the Machine Learning Notebook.

Standard ConverSight Notebook : The Standard ConverSight Notebook serves as a foundational tool, encompassing essential packages for executing tasks, flows, CS App and conducting fundamental analytical operations.

Machine Learning Notebook : The Machine Learning Notebook is specifically crafted to empower users with an extensive set of packages catering to the intricate requirements of machine learning and advanced analytics.

Users can create custom Notebook by integrating a custom package or a ConverSight verified package into the Base Standard ConverSight Notebook, resulting in the creation of a custom Standard ConverSight Notebook. Likewise, incorporating custom packages into the Machine Learning Notebook facilitates the development of a customized Machine Learning Notebook, catering to specific requirements.

Notebook

NOTE
Editing or deleting the default Standard ConverSight Notebook and Machine Learning Notebook is restricted.

To work and know more about Custom Notebooks you can refer Custom Notebook.

• To initiate the Notebook, you can simply click on icon located in the action field of the associated Notebook.

Start a Notebook

Upon clicking the icon, a Notebook instance is initiated, triggering a request to the Jupyter Hub repository, accompanied session details. Jupyter Hub then verifies your access to the Notebook and presents you with a list of accessible Notebooks if you have multiple permissions. Alternatively, if you have a single Notebook access, the server will launch it automatically.

Jupyter Hub

It is worth noting that there are different types of Notebook available: The type of Notebook assigned to a user is determined by their configuration whether its low-level or medium -level or high memory load processes.

After the configuration is finished, a dedicated service will commence the Notebook session under your personalized username and the session will expire after 4 hours. Any actions performed or files saved will be exclusively attributed to your user, leaving no impact on other users.Any library installed by the user will be deleted once the session gets over, if a user needs a library to be installed permanently then the user should request DevOps team to install it permanently.

Noteboook

NOTE
Any packages installed by user during a session will only be stored for a maximum of four hours. Once the server is stopped, they will be deleted.

Creating a New Notebook

To create a new Notebook, click the New button in the upper right corner of the Notebook interface and select the type of notebook you want to create (e.g. Python). This will create a new Notebook and open it in a new tab.

New Noteboook

Working with Cells

Notebook is made up of cells. Each cell can contain code, markdown or raw text. To create a new cell, click the + button in the toolbar or use the keyboard shortcut (Shift + Enter). To execute a cell, you can either click the Run button in the toolbar or use the keyboard shortcut (Ctrl + Enter).

New Cell

Saving, Uploading and Downloading a Notebook

Notebook can be saved to your local computer. To save a notebook, click the File menu and select Save.

Save

By clicking on the Upload icon, the user can upload the notebook.

Update

To download a Notebook from Jupyter Notebook, simply right-click on the Notebook file and select the Download option, which will initiate the download process.

Download

ConverSight Library

The ConverSight library is a specialized resource within ConverSight, offering a comprehensive set of capabilities. With the ConverSight library, you can perform a wide array of tasks, such as creating and managing tasks and workflows, enlisting them in a sequential task and workflow repository, establishing connections between datasets, examining dataset specifics and metadata, executing diverse operations with datasets, accessing group and role information and executing queries.

In ConverSight, the term Dataset denotes a meticulously organized and structured collection of data, tailor-made for the ConverSight platform. The ConverSight library empowers you to conduct numerous functions in association with these datasets.

NOTE
You can conveniently view the necessary arguments for a specific function by pressing both the Shift and Tab keys simultaneously.

Connecting Datasets

Within the ConverSight Notebook, users possess the ability to establish a connection to their datasets using the datasetID, thereby granting them the capacity to perform a diverse array of functions.

Below are various functions that pertain to datasets:

• Listing Objects

• Schema

• Querying in DataFrame Format

To commence, import the necessary libraries and packages. Access the dataset by utilizing its datasetID, as shown in the following code example.

 from conversight import Dataset
 ds = Dataset("DatasetID")
 # This will import all the dataset from the conversight Library.

Importing Dataset

Listing Objects

The tables in the dataset are referred to as objects. To obtain a detailed list of objects and their respective properties within the dataset, you can utilize the objects.list() method.

 ds.objects.list()
 # This will display the table names in the selected dataset.

Listing Objects

Schema

To access additional details about the dataset schema, you can determine its activation status using the active method within the schema. This method returns true if the dataset is active and false if it is inactive. When the current method in schema yields a value and the active schema confirms its validity, it indicates that the current schema is operational. Consequently, Athena queries will be executed within this schema.

 ds.connectors.connectorInfo.connectionData.schema.current()
 # This will display the Current Schema of the dataset.

 ds.connectors.connectorInfo.connectionData.schema.active()
 # This displays the Active Schema that is in use.

Dataset Current and Active Schema

Querying in DataFrame Format

To view the query in Data Frame format, make use of the sqlDf() method. The argument must be in the format of an SQL Query. Please be aware that each SQL query is tailored to the specific dataset. The limit parameter plays a crucial role in controlling the number of displayed rows, ensuring a concise presentation of results. If the limit is not applied, the entire Data Frame will be presented.

 df = ds.sqlDf("""select * from RetailSales limit 5""")
 # This displays the query in DataFrame Format with the limit parameter.

Limited DataFrame

 df = ds.sqlDf("""select * from RetailSales""")
 # This displays the query in DataFrame Format without the limit paramter.

Unlimited DataFrame

Roles

In ConverSight Notebook, through the use of the “Role” functionality, you have the capability to allocate particular data access permissions to a user.

Available Methods in Roles

Pressing the Tab button on an object or function displays a list of the available methods. List of available methods for roles are shown in the below screenshot.

Roles

Get

The roles method retrieves information about the roles associated with a dataset, including the role ID, name, description, dataset and role filter. This method is useful for understanding the permissions and access of the roles. By using the roles method and the get function, developers and data analysts can quickly retrieve important information about the roles defined within a dataset.

ds.roles.get()

Roles

Create

The roles.create() method can be used to create roles using the Notebook in ConverSight.

Create Roles Help Guide

 ds.roles.create('Users','creatingroles','dimensions','RetailSales.revenue','is','1','100')
 # This creates the role required.

Created Roles

Update

The roles.update() method can be used to update any existing roles you have created using Notebook in ConverSight.

 ds.roles.update('650bde77','users','trial','dimensions','RetailSales.revenue','is','1','50')
 # This updates the created role.

Updated Roles

Delete

Any created roles can be deleted by Data Admin only.This can be done by using the roles.delete() method.

Deleting Roles Help Guide

 ds.roles.delete('650bde77')
 # This deletes the role.

Deleting Roles

Groups

The groups method retrieves information about the groups associated with a dataset, including the group ID, name, description, dataset and group filter. This method is useful for understanding the permissions and access of the groups.

Available Methods in Groups

Pressing the Tab button on an object or function displays a list of the available methods. List of available methods for roles are shown in the below screenshot.

Groups Methods

Get

The group method contains the get() function, which displays the group ID, group name, description, dataset, column list, settings and form list. The user can use this method to understand about the permission and access given to the database.

ds.groups.get()

Groups

Create

Users have the ability to create groups using the Notebook in ConverSight. This can be performed by using the groups.create() method. The user can refer to the help guide to determine the necessary arguments required for utilizing the method.

Creating Groups Help Guide

 ds.groups.create('Usergroup','Sample creation',['RetailSales.revenue'],None,None)
 # This will create a group.

Created Groups

In the upcoming sections, we’ll delve into the Task and Flows, which will be explored using a Notebook. Let’s have a conversation about this!