Recently, Artificial Intelligence (AI) has been gaining popularity at breakneck speed.
OpenAI’s ChatGPT was a breakthrough in Artificial Intelligence, and the enthusiasm was huge. ChatGPT triggered a trend towards AI applications that many companies followed. You read and hear about AI everywhere. Videos and images of celebrities performing strange dance moves appear, or you hear interviews and songs by artists who have “actually” been dead for several years.
The exact functioning of Artificial Intelligence will be explained below. As developers, we are used to thinking in categories and breaking problems down into small steps, and this is exactly how all Artificial Intelligence works. They are based on so-called models that have been trained for their respective areas of application.
Example:
If you ask ChatGPT for a picture, a separate model such as DALL-E must be used.
If you ask for a poem, ChatGPT uses a language model.
ChatGPT is nothing more than a ChatBot that uses different models for the task at hand.
This is where I would like to start and program a small Artificial Intelligence that classifies a customer review, which is available as free text, as positive or negative.
As a .NET developer, Microsoft offers a low-threshold entry into the world of machine learning with ML.NET. This free open-source framework makes it easy to integrate ML models into new and existing .NET applications – without any knowledge of Python, R, or other ML libraries. ML.NET already contains many ready-made models for common use cases and can be easily integrated into the existing .NET app lifecycle.
Basics of Machine Learning
The term “machine learning” describes the approach of teaching computers to learn patterns and relationships from data instead of explicitly programming them with rules and program logic. The aim is to enable the machine to “learn” independently from sample data and apply this knowledge to solve similar problems.
There are essentially three types of learning:
1. Supervised learning
Supervised learning is a method of machine learning in which a model is provided with training data consisting of input data (e.g. texts) and the corresponding desired outputs (e.g. classifications such as ‘positive’ or ‘negative’). Using these examples, the system learns to recognize patterns that relate the inputs to the outputs. This enables it to later predict correct outputs for new, unseen inputs. A classic application example is object recognition in images.
Example:
An application example for supervised learning with ML.NET could be the classification of product reviews as ‘positive’ or ‘negative’. The procedure would be as follows:
At this point, we have a trained model that can classify new reviews.
2. Unsupervised learning
Unsupervised learning is a machine learning approach in which the model is only provided with unlabelled input data, e.g. text or images, without any associated target variables or classifications. The system must independently recognize structures, patterns, and correlations in this raw data. A typical area of application is clustering, in which similar input data is summarised into groups. Possible examples are the identification of customer segments based on marketing data or the grouping of news articles according to subject areas when analyzing text.
Example:
Unsupervised learning can be used with ML.NET for text clustering, for example:
In this way, for example, news articles can be automatically categorized into thematic clusters such as politics, business, sports, etc.
ML.NET focuses mainly on supervised and partially unsupervised learning. ML.NET does not currently offer any specific support for reinforcement learning, in which an AI is trained through rewards in a simulated environment. This learning paradigm is used in particular for strategy-finding tasks, such as games or the optimization of processes.
3. Reinforcement learning
Reinforcement learning is a learning paradigm in which Artificial Intelligence is trained by trial and error in a simulated environment. In contrast to supervised learning, there are no complete training examples with input-output pairs. Instead, the system receives a score (reward) for each action, which indicates how “good” this action was. By maximizing the accumulated rewards, the AI learns to find the optimal strategy for a task. This principle of reinforcement learning is used in particular for tasks that require a complex strategy to be found. Examples include board games such as chess or Go, but also applications for robot control, process optimization, or autonomous driving.
Example:
A classic application example is a chess player agent. Here the environment could be a chessboard, possible actions are move movements. The agent then receives an evaluation for each move, e.g. +1 for a sure pawn win, -5 for an impending piece loss. Through many training games, the AI learns which moves to lead to a win in the long term and thus maximizes its cumulative reward.
Set up the development environment:
To get started with ML.NET, we need the .NET Core or .NET Framework environment in version 4.6.1 or higher. ML.NET can be used in Visual Studio, Visual Studio Code, or the command line.
The first step is to install the required NuGet packages of ML.NET:
This main package contains basic functionality such as the ML.NET context, data transformations, and catalogs with models and tasks. For additional functionality, other packages such as Microsoft.ML.Vision for image processing or Microsoft.ML.Recommender for recommender systems can be added.
After installation, we can create a .NET Core or .NET Framework console application in Visual Studio and start development.
Optionally, integrated tools such as the Model Builder interface in Visual Studio or the ML.NET CLI can be used to visualize data and create models. I’ll come back to this later.
Creating the first ML model
In the following, the basic concepts of ML.NET are explained step by step using an example:
The classification of customer reviews is based on free text.
The result should be to categorize these ratings as positive or negative.
This is a classic supervised learning problem.
We create a new C# console application and import the ML.NET NuGet packages.
Next, we load the data set into the working memory. In our example, we need a TSV (tab-separated file) file for this. This is represented by a simple class in our console application:
This class stands for each line within the TSV file.
Now we add properties for each column of the TSV data:
Unfortunately, this is not enough to load this TSV file and teach ML.Net how to reference this file.
We need attributes for this: LoadColumn plus the index of the column.
This is a simple mapping from the ReviewText property to the TSV file ReviewText column.
Now let’s take care of the storage paths for the files and save them in our application:
First, we create the path to our training file, the TSV file (Review.tsv).
We will use this file to train the model that we want to use for classification.
Now we define the path to the model that we want to train. This model is saved as a ZIP file.
Now we need a variable for our MLContext. The MLContext is comparable to an EF DbContext. So, whenever you want to work with machine learning, you have to interact with the MLContext, similar to the DBContext in EF Core.
Now we are ready to load the data. We can manipulate it as we need it to train the model according to our requirements.
Now we build our model:
For this, we need a specific type from ML.NET. The IDataView:
Here I can now use a function from the MLContext called LoadFromTextFile. `<ReviewData>` is now used to project the data into our initially created class. The path to the training file must now be passed to this function. As a bonus, we can specify further options here, such as the separator from the TSV file and whether our training file has a header or not (we have a header, so I set this parameter to true).
Now that we have loaded the data into a DataView, we can start preprocessing. Preprocessing is a means of creating what is called a pipeline. This can later be used to perform predictions when we have incoming data (e.g. our customer score). To do this, we need to extract the data from the data view we have created (IDataView) and then transform it so that it can be used as part of the machine-learning process.
Now let’s create our pipeline. This has a specific return type: IEstimator<ITransformer>:
Context.Transforms.Text.FeaturizeText is a text transformation component in ML.NET that is used to convert text data into numerical features.
“Features” is the new column name that contains the resulting numerical characteristics.
“ReviewText” is the name of the column from the TSV file that contains the text to be converted into numerical characteristics.
Now the training of the model begins:
`.Append` adds the next step to the pipeline. The model is trained with `Context.BinaryClassification.Trainers.SdcaLogisticRegression`, which defines the algorithm with which the model is to be trained – in this case the SdcaLogisticRegression algorithm. “Label” is the column or the name of the column that represents the target variable to be predicted. In our example, it is the classification of whether this evaluation is positive or negative.
Let’s summarise this again:
This command builds a pipeline that performs the following steps:
text featurisation: the text from the “ReviewText” column is converted into numerical features and stored in the new “Features” column.
model training: The numerical features in the “Feature” column are used to train a binary classification model with the target variable in the “Label” column.
Now we have created our pipeline and can start training the model.
We use the `pipeline.Fit(…)` command to train the model on the data from the TSV file. The model is then stored in memory.
However, to save the model and not have to create and retrain it each time, we can use the `Save()` method. This allows us to save the trained model in a specific file path. In this way, we can simply load and use the model when needed without having to retrain it each time.
There is an option in MlContext where we can find the Save() function:
Firstly, we transfer the trained model and the schema from the IDataView from the TSV file, and then the path where we want to save the model.
We have now trained our model and saved it to the hard disc.
Now we need another class that represents the trained DataSet, i.e. the results from our model (prediction class). To do this, we create another class that has a similar structure to the class for the DataSet from the TSV file (ReviewData).
The attribute defines that the name of the “PredictedLabel” column is to be used in the MLContext to read the predicted value there.
To connect the model to the prediction class, we now need a PipelineEngine. We create this engine again from the MLContext.
The `CreatePredictionEngine` method is used to create predictions based on the previously trained model.
We use the generic types TSrc (Source) and TDst (Destination) for our input and output classes, “ReviewData” and “ReviewPrediction” (“Good rating”, “Positiv”).
The individual components and their interaction should now be understandable: Why we need to convert texts into numerical features, why we need to prepare data so that the computer can understand it, and that our “ReviewData” class is our source class and “ReviewDataPrediction” is our target class.
Now that we have completed the preparation steps, we can move on to the prediction phase.
Let’s now take care of the code that performs the prediction and returns the expected target variable.
In our example, the predicted result is a Boolean value.
The first step is to enter a score, which we then write to our “ReviewData” input class.
The variable “ReviewText” is a string variable that contains our review.
We now pass this input variable of type “ReviewData” to the pipeline that we have just created.
As a result, we receive our “ReviewPrediction” output class and can now access the target variable from the model via the variable.
f the model had good training data, we should now get the correct predicted results.
It seems to work.
Of course, this is only a small example, and the model is not perfect, but if you train it well, it should ideally always return correct results.
The following extensions have been added to the example to retrain the model if inputs were incorrect or unknown.
When starting the application, you can choose between adding new data to the model and direct training or you can create new comments.
In the next article, I will take a closer look at image recognition and prediction.
Summary:
Although this article is limited to text classification, on closer inspection, and with a little creativity, the approach described opens up numerous possibilities. For example, the recognition of a certain classification could be used to trigger further processes automatically.
Overall, it can be said that Artificial Intelligence has enormous potential to support us in many areas of everyday life and to optimize processes. Text classification is just one example of the many possible applications. It can be assumed that more and more areas will benefit from the possibilities of AI in the future and that this will open up new opportunities for companies and private individuals alike.
To fully exploit this potential, it is important that we are open to new technologies and that we address the possibilities and limitations of AI. This is the only way we can ensure that Artificial Intelligence is used responsibly and for the benefit of all in the future.
Demo:
Benjamin Witt
Macrix Lead Developer