Monitor Equipment State of Health Using Drift-Aware Learning on the Cloud
This example describes the set up necessary to run the deployed version of the Monitor Equipment State of Health Using Drift-Aware Learning example on the cloud. The topic shows how to automate the process of monitoring the state of health for a cooling system using an incremental drift-aware learning model using the infrastructure in the next figure. This example requires Statistics and Machine Learning Toolbox™, MATLAB® Compiler SDK™, MATLAB Production Server™, and MATLAB Web App Server™.
The following figure shows the architecture for drift-aware learning. The architecture involves two analytics functions
processLabeledData.ctf that are deployed on the MATLAB Production Server (MPS).
processUnlabeledData.ctf loads the model, uses it to predict the health of the equipment, and writes the sensor readings and predictions to the output topic
processedLabeledData.ctf reads the labeled data from input topic labeled and uses the labeled data to check the model for drift. It then outputs the labeled data, along with the drift diagnostic information, into the output topic
A dashboard deployed on the Web App Server then reads the processed data. The dashboard enables to visually inspect the sensor readings and model performance.
Redis server enables the updated model to be shared between the labeled and unlabeled CTFs.
Upload Sensor Readings
Start Kafka server from a dockerized container.
To upload sensor readings, you must first create the Kafka topics:
processedLabeled. You must have a Kafka server running at
kafka.host.com at port number 9092 for this example. For more details on uploading data using a Kafka server, see Streaming Data Framework for MATLAB Production Server Basics (MATLAB Production Server).
Upload readings via
Similar to the Monitor Equipment State of Health Using Drift-Aware Learning example, upload the unlabeled and labeled data (provided in
LabeledData.mat) to the Kafka server using the
labeled topics via the
uploadApp2, also provided as a supporting file. The
uploadApp2 deployable archive runs on the Web App Server. To learn more about how to set up a deployable archive on Web App Server, refer to the instructions in Create Web App (MATLAB Compiler).
After deploying the
uploadApp2, click Start Streaming. The app uploads 60,000 unlabeled sensor readings and 6000 labeled sensor readings, stored in the files
labeledData.mat, respectively. Each sensor reading is sampled every 0.1 second.
The Upload Status light turns red while the app uploads the unlabeled and labeled data. When the upload is complete, the status light turns green. The Kafka server now contains data that can be accessed by anyone responsible for maintaining a cooling system.
Monitor Cooling System
To start monitoring the cooling system, first you must create the deployable archives
processLabeledData.ctf. To deploy these two archives, you must have the MATLAB Production Server running. Start with
processUnlabeledData.ctf first since it loads the drift-aware learning model into the Redis cache. The model is a warmed-up incremental drift-aware learner with incremental Naive Bayes classification model as the base learner.
Create deployable archive
processUnlabeledData.ctf and move it to MPS.
processUnlabeledData.ctf into a deployable archive by running the below script in the supporting file
kafkaHost = "kafka.host.com"; kafkaPort = 9092; inputKS = kafkaStream(kafkaHost,kafkaPort,"unlabeled","Rows",1200,"RequestTimeout", 60,'TimestampResolution','Seconds'); outputKS = kafkaStream(kafkaHost,kafkaPort,"processedUnlabeled","Rows",1200,"RequestTimeout",60); archive = streamingDataCompiler("processUnlabeledData",inputKS,outputKS,StateStore="LocalRedis",InitialState="initUnlabeledData");
Start Kafka connector.
Start Kafka connector at port 1234 for
powershell -executionPolicy bypass -File kafka-connector-start.ps1 -out out.log -err error.log -c collector.properties -k kafka.properties -p "1234"
chmod +x kafka-connector-start.sh
!./kafka-connector-start.sh -out out.log -err error.log -c collector.properties -k kafka.properties -p "1234"
Starting the Kafka connector initializes
processUnlabeledData.ctf, that is, uploads the warm incremental drift-aware learner model and the
processUnlabeledData deployable archive listens for data in the input topic
Create deployable archive
processLabeledData and move it to MPS.
Repeat the above steps to create and deploy
processLabeledData.ctf using the supporting file
createCTFLabeled.m and start the Kafka connector at port 1235 for the
processLabeledData archive begins to read the labeled data and updates the model parameters in the event of a drift in the data distribution. The Redis cache allows for the model to be shared between the two deployable archives.
Open the dashboard app hosted on Web App Server.
The output topics
processedLabeled contain the processed data. Once both
processLabeledData.ctf are processed, the
processedUnlabeled topic contains the sensor readings along with the model predictions, whereas the
processLabeled topic contains the sensor readings along with the model drift diagnostic information.
dashboard2 app reads the processed data from the
processedLabeled topics. The app runs on the Web App Server, deploy the
dashboard2 app using the MATLAB Web App server by following the instructions in Create Web App (MATLAB Compiler).
Once deployed, the
dashboard2 app is ready to start streaming the data uploaded to the Kafka server. You can initiate the streaming process by setting Streaming Options to
View the Sensor Reading tab to inspect the sensor readings of the data received in the previous two minutes.
View the Model Performance tab to monitor the drift in the data.