Saturday, August 26, 2023

How to Develop Alteryx Custom Tool

Post Index  

2023-08-26


How to Develop Alteryx Custom Tool

with the help of Platform SDK

AYX UI SDK, AYX Python SDK v2, AYX Plugin CLI



The platform SDK allows you to develop Alteryx custom tools and plugins.  Alteryx provides a lot of amazing tools but it does not come out of the box for everything.  Just like my previous posts of QVD Input Tool and QVD Output Tool, Alteryx does not come out of the box to have the capability to manipulate QVD files.  In the past, there are Engine and HTML GUI SDKs.  As mentioned in the Alteryx platform SDK help, the outdated technology is limited for extension.  There is a need to use the latest technology that can extend the opportunities for improvement.

For details of Platform SDK, you may refer to the help. This article will focus on the basic to let you understand how to develop a simple Alteryx Custom Tool.




Environment Setup

For basic requirements, please refer to AYX Python SKD v2 help page.

Before you can start the development, there are a few steps required.

1. Install Minicoda3 (download from the site and install).


2. Open Anacoda Prompt (click on the Windows button and search it)


3. Create a virtual environment with the command
    conda create -n [ENV_NAME] python=3.8.5

e.g. conda create -n test_env python=3.8.5

4. Activate the environment.
    conda activate [ENV_NAME]

e.g. conda activate test_env

4. Install the AYX Python SDK.
    pip install ayx-python-sdk


5. Install the AYX Plugin CLI
    pip install ayx-plugin-cli


6. Create (or initialize) the AYX Plugin Workspace
    ayx_plugin_cli sdk-workspace-init

    A few questions to answer to help initialize the configuration files.

    Package Name: [The name of the package]
    Tool Category [Python SDK Examples]: [The category of the tools]
    Description []: Description of the package.
    Author []: Who is the author of the package.
    Company []: The company name.
    Backend Language (python): python

7. Install NodeJS.  It is required to developer UI for the tool.
    conda install -c anaconda nodejs=16.13.1




7. Create the AYX Plugin
    ayx_plugin_cli create-ayx_plugin --use-ui
    * --use-ui must be explicitly input to make sure the UI part is created by the plugin CLI.




    A few questions to answer to help initialize the configuration files.

Tool Name: [The name of the tool]
Tool Type (input, multiple-inputs, multiple-outputs, optional, output, single-input-single-output, multi-connection-input-anchor) [single-   input-single-output]: [the type of the tool]
Description []: [description of the tool]
Tool Version [1.0]: [version of the tool]
DCM Namespace []:  [DCM Namespace to be used]

Once it is done, you will have a folder structure created as below.

Backend Development (Python)

After the initialization of the environment, workspace and tool, there is a backend file created in the path \backend\ayx_plugins.

Open this using an editor or any IDE to start the tool development.  There are four important functions:

def __init__(self, provider: AMPProviderV2) -> None:
This is to initialize the tools.  Usually, to initialize the variable to be used in the backend code.

def on_incoming_connection_complete(self, anchor: namedtuple) -> None:
When there is an incoming anchor complete, this will be called.  Input tool does not required to handle this.

def on_record_batch(self, batch: "Table", anchor: namedtuple) -> None:
This is to handle the record provided in batches.  Input tool does not required to manipulate this.

def on_complete(self) -> None:
Once the batches have been manipulated, this is to free up the resources and also finalize the completion.  All logics happen here for input tool.

For each function, the comment is already in the backend plugin file. 

Frontend Development (Reactjs)

For front-end development, it should locate to the path \ui\TestInputTool\src.  There is a file called index.tsx.  This is the file to be updated.

The content is reactjs.

More UI help can be found in the Alteryx UI help.


Create the YXI and Deploy to Alteryx Designer

Once the frontend, backend, configuration (like icon, description, version, etc) are completed, it can be packaged to YXI file to share and also install into Alteryx Designer.  It is simply running the command:
ayx_plugin_cli designer-install --use-ui
* --use-ui must be specified explicitly if UI component is involved.

It will prompt a question.
Install Type (user, admin) [user]: [user means to install user location that only for single user while admin means to install into admin location that allows all users in the machine to use]

user location: C:\Users\[user]\AppData\Roaming\Alteryx\Tools
admin location: C:\ProgramData\Alteryx\Tools


Test in Designer

Open up the Alteryx designer, the tool palette contains a new category named "Test Tools" and the Test Input Tool can be selected.  Once this executed, there is a sample result.



This concludes how to develop an Alteryx Custom Tool.  For details about python code, reactjs, etc, leave me message if further discussion is needed.


Thank you for reading.  I hope it help you.  Just leave a message if you have any discussion/share want to make.



Thursday, August 17, 2023

Alteryx QVD Output Tool - Prototype

 Post Index

2023-08-17


Alteryx QVD Output Tool

Alteryx Custom Tool




In the previous post, the Alteryx QVD Input Tool prototype is shared and following in this post, the Alteryx QVD Output Tool will be introduced.

It reads the data from Alteryx and then convert into QVD, i.e. convert each column into a list of symbols and symbol indexes and then compact each record  by the symbol index into the least bits required to store the record data.  XML information is saved in memory during the processes.  Once everything is ready, it flushes out the XML, symbols as well as the records.


Alteryx QVD Output Tool

The Alteryx QVD Input Tool is very simple.  It just takes in a QVD file and read all the content then convert it as an Alteryx output stream.  The input UI is as below.



* the new Alteryx SDK is now using reactjs where it is not possible to get through the security to get the full path.  Thus, there is no button to pop up a dialog to ask for file location.  Instead, there is only a textfield for inputting the path.  If you have any clue to get this through, it is welcome.  The prototype is hoping to show the possibility to integrate with QVD files.


The prototype

If you hope to try it, you can download it in my github.  https://github.com/kongson-cheung/Alteryx-QVD-Tools/blob/main/yxi/QVD%20Tools_v1.1.yxi

I have share the core files to create this Alteryx QVD Output Tool.  Since the SDK includes a large number of files, I did not upload them all.  If you need any help, feel free to drop me a message.


* Note: this is still very early version of prototype.  It still requires a number of improvements for intensive use.


Next

I will try summarize how to develop the Custom Alteryx Tool.


Thank you for reading.  I hope it help you.  Appreciated your sharing if you have any discussion/share want to make.



Monday, August 7, 2023

Alteryx QVD Input Tool - Prototype

 Post Index

2023-08-07


Alteryx QVD Input Tool

Alteryx Custom Tool


With the findings in the previous post (QlikView Data File (QVD) - Reverse Engineering), I have developed a prototype of the Alteryx QVD input tool.  Alteryx is an extremely good tool for data wrangling and contains a bundle of tools that allow simple data transformation and predictive analysis.  However, it does not have the ability to deal with QVD.  It only allows to read and write QVX.

Alteryx, in fact, is a good upfront data stream for QlikView and Qlik Sense.  It can help business to make clear use of data with drag and drop capability to explore, transform and try new business logics.  QVD integration would be benefitial for Qlik at the lower data stream in the data cycle.   Sound like advertisement but it is real project experience to conclude this.  QVX does not work great with heavy usage.  The performance is similar to CSV.  Still, QVD is the best for Qlik.


Alteryx QVD Input Tool

The Alteryx QVD Input Tool is very simple.  It just takes in a QVD file and read all the content then convert it as an Alteryx output stream.  The input UI is as below.


* the new Alteryx SDK is now using reactjs where it is not possible to get through the security to get the full path.  Thus, there is no button to pop up a dialog to ask for file location.  Instead, there is only a textfield for inputting the path.  If you have any clue to get this through, it is welcome.  The prototype is hoping to show the possibility to integrate with QVD files.


An Example

Taking a QVD file as an example.



The QVD file contains 2 columns named Num and Text.  It has total number of 4 records.  In Alteryx, the result runs as below.

If you hope to try it, you can download it in my github.  https://github.com/kongson-cheung/Alteryx-QVD-Tools/blob/main/yxi/QVD%20Tools_v1.0.yxi

I have share the core files to create this Alteryx QVD Input Tool.  Since the SDK includes a large number of files, I did not upload them all.  If you need any help, feel free to drop me a message.


* Note: this is still very early version of prototype.  It still requires a number of improvements for intensive use.


Next

The Alteryx QVD Input Tool is made up by Platform SDK, UI SDK, Python SDK.  This is a first prototype for reading QVD.  I am exploring a prototype of Alteryx QVD Output Tool that write QVD as output.


Thank you for reading.  I hope it help you.  Appreciated your sharing if you have any discussion/share want to make.