Portable Models and Model Insights
Note
As of Release 6.7 clients will no longer be able to create new Predictive Coding sessions. Existing Predictive coding sessions are not disabled, they can continue to be used in the near term. It is highly recommended that clients use CMML with ACS as a replacement for PC.
Portable Models
Portable Model Overview
Brainspace’s Predictive Coding and Continuous Multimodal Learning (CMML) classifiers support generating a predictive model on a dataset after coding example documents. The predictive model can then be used on later classifiers (usually in different datasets) to score all of the documents in the classifier to prioritize and classify them.
A portable model is a *.csv (comma-separated values) file containing a simplified version of a CMML predictive model. Each line in the *.csv file is a feature description followed by a portable weight between -100 and 100.
You can create a portable model in three ways:
Exporting a predictive model from a classifier within an active dataset
Starting with an existing portable model and editing it
Manually creating a *.csv file in the proper format
When exporting an existing predictive model as a portable model, only the most influential features are retained and the coefficients are scaled and rounded between -100 and 100. An exported portable model can be either saved outside of Brainspace as a *.csv file or within Brainspace in the portable model library, managed in the Administration panel.
An existing portable model in *.csv file form can be edited, or curated, before importing it into a new dataset. Editing can drop features, change the weight of features, and add features. Features can be words or phrases, as well as metadata values described by feature descriptions.
Since a portable model is simply a *.csv file with a particular format, it is also possible to create a portable model file manually , without beginning from an existing exported portable model.
An imported portable model will be converted to a predictive model. The predictive model can then be used just as if it was trained on that dataset. It can also be updated with training data from the dataset to further improve it.
Portable Model File Formatting and Editing
A portable model is a *.csv (comma-separated values) file. The first row of the file is a header with the column headings “term” and “weight.” The remaining lines each contain a single pair consisting of a feature description in the first column and a portable weight in the second column. Below are examples of feature-weight pairs:
purple, +5
augmented intelligence, +7
échantillon aléatoire, +2
机器学习, +3
“[“”cc””,””fred jimes <jimes@foo.org>””]”, +1
“[“”emailclient””,””outlook””]”, +2
“[“”created-year-month-day-hour””,””2001120412″”]”, -3
The first four examples are textual features, while the last three are metadata features. As shown above, the feature description for a term (a textual feature) is just the term (word or phrase) itself. You can enter terms manually in a *.csv file or convert an existing lists of keywords to *.csv format.
Metadata feature descriptions require more care. It is important both to know what derived features are present in your dataset of interest, and to observe the proper formatting of the JSON array used for metadata feature descriptions. When using metadata features in a portable model, we recommend starting with an exported predictive model from the dataset of interest, or one with the same fields and import configuration.
Portable weights must be integers between -100 and 100, including 0. There should be no decimal point, and scientific notation should not be used (so write 100, not 1E+2).
Portable Model Character Encoding
Portable model files use the UTF-8 character encoding. The usual 7-bit ASCII encoding is equivalent to UTF-8 for the characters typically used in English, so you can usually ignore character encoding issues when editing portable model files that contain only English.
For portable model files that contain non-ASCII UTF-8 characters, care needs to be taken to both import the portable model *.csv file using UTF-8 encoding and to export the *.csv file preserving the UTF-8 encoding. This may be challenging in some versions of Microsoft Excel, for instance. You may want to consider using a text editor or *.csv file editor with explicit UTF-8 support.
New Portable Model
When creating a new portable model, provide a name, select a *.csv file to upload, and optionally select which user groups can access the portable model.
Create a Portable Model from a CMML Classifier
After creating a CMML classifier and running training rounds, you can convert any completed training round to a portable model.
To create a portable model from a CMML classifier training round:
Click the Supervised Learning tab.
The Supervised Learning screen will open.
Click a Classifier card.
The Classifier screen will open.
In the Training pane, click the Portable Model Actions icon:
The Portable Model Actions dialog will open.
To save the portable model to the Brainspace library, Click the Save to Portable Models button.
The Save Portable Model dialog will open.
Type a name for the portable model.
Toggle one or more Portable Model Groups switches to the On position.
Click the Save button.
After creating the portable model, you can use it to create a new CMML classifier, or you can download the Portable Model as a *.csv file.
Download a Portable Model as a *.csv File
After running CMML classifier training rounds, you can download a *.csv file for any of the training rounds.
To download a Portable Modal as a *.csv file:
Click the Supervised Learning tab.
The Supervised Learning screen will open.
Click a Classifier card.
The Classifier screen will open.
In the Training pane, click the Portable Model Actions icon:
The Portable Model Actions dialog will open.
To save the portable model to the Brainspace library, click the Save to Portable Models button.
The Save Portable Model dialog will open.
Type a name for the portable model.
Toggle one or more Portable Model Groups switches to the On position.
Click the Download button.
Model Insights
Model Insights Overview
Model Insights provides details about the terms and phrases that are influencing a classifier during training. You must create at least two training rounds for a classifier before using Model Insights.
Insights compares the features and weights of a selected portable model to those of a prior portable model or optionally to a portable model.
Note
Rank comparisons are based on the sort order of the features.
Insights Dialog
The Insights dialog accessed from a training round's Show Insights button includes the following features:
Select a model from an earlier round to compare with the selected training round. In this example, we are comparing round 4 to round 5.
Toggle the switch to add portable models to the To dropdown menu to compare with the current model.
Filter the terms list for a specific term or text-string in the training round selected for comparison.
Download the Insights comparison report for the rounds selected to compare.
Click the filter buttons to view terms that were added (relatively more predictive of target category), terms that were removed (relatively less predictive of target category), terms that increased or decreased in rank, and terms that have not changed in rank.
To view impactful terms or text-strings, click the column header to sort the columns by increasing or decreasing values:
Prev. Rank and Prev. Wt: Impact of the term or text-string in the model in the (compare) To column
Rank and Weight: Current impact of term or text-string in the selected model
Rank Diff. and Wt Diff: Change in impact of term or text-string between the two models
Note
Rank comparisons are based on the sort order in the portable model file.
View Model Insights for Portable Models
After creating two or more portable models in Brainspace, you can compare rank and weight values between two classifiers. You can also use Model Insights to compare two training rounds in a single CMML classifier.
To view Model Insights information for classifiers:
Create at least two portable models.
Click the Supervised Learning tab.
The Supervised Learning screen will open.
Click the Insights icon:
The Insights dialog will open.
Click a choice in the Compare and To dropdown menus.
The Insights dialog will refresh to display terms and associated ranks and weights for the Insights comparison, with the Added Terms selected by default in the Filters field.
You can compare results for one or more of the filters by clicking and unclicking additional Filter buttons. You can also search for terms and text-strings to refine filter results and download a Insights comparison report to record and preserve the comparison results.
View Model Insights for Training Rounds
After creating at least two CMML classifier or control set training rounds, you can compare training data for a specific range of training rounds.
To view Model Insights information:
Create a classifier, and then run at least two training rounds.
Click the Supervised Learning tab.
The Supervised Learning screen will open.
Click a Classifier card.
The Classifier screen will open.
After at least two training rounds complete, click the Show Insights button.
The Insights dialog will open.
Terms with the plus (+) icon in the Change column have been added to the model. Terms with the minus (-) icon in the Change column have been removed from the model.
Compare Training Rounds
To compare the current training round with an earlier training round, navigate to the Insights dialog, click the To dropdown arrow, and then click a training round in the list:
Compare a Training Round with a Portable Model
To compare the current training round with a Portable Model, navigate to the Insights dialog, and then toggle the Compare with Portable Models switch to the On (green) position:
Download Model Insights Comparison
To download an Insights comparison to a *.csv file, navigate to the Insights dialog, and then click the Download Insights Comparison icon: