Categories &

Functions List

Function Reference: fitcnet

statistics: Mdl = fitcnet (X, Y)
statistics: Mdl = fitcnet (…, name, value)

Fit a Neural Network classification model.

Mdl = fitcnet (X, Y) returns a Neural Network classification model, Mdl, with X being the predictor data, and Y the class labels of observations in X.

  • X must be a N×P numeric matrix of predictor data where rows correspond to observations and columns correspond to features or variables.
  • Y is N×1 matrix or cell matrix containing the class labels of corresponding predictor data in X. Y can contain any type of categorical data. Y must have same numbers of rows as X.

Mdl = fitcnet (…, name, value) returns a Neural Network classification model with additional options specified by Name-Value pair arguments listed below.

Model Parameters

NameValue
'Standardize'A boolean flag indicating whether the data in X should be standardized prior to training.
'PredictorNames'A cell array of character vectors specifying the predictor variable names. The variable names are assumed to be in the same order as they appear in the training data X.
'ResponseName'A character vector specifying the name of the response variable.
'ClassNames'Names of the classes in the class labels, Y, used for fitting the Neural Network model. ClassNames are of the same type as the class labels in Y.
'Prior'A numeric vector specifying the prior probabilities for each class. The order of the elements in Prior corresponds to the order of the classes in ClassNames.
'LayerSizes'A vector of positive integers that defines the sizes of the fully connected layers in the neural network model. Each element in LayerSizes corresponds to the number of outputs for the respective fully connected layer in the neural network model. The default value is 10.
'LearningRate'A positive scalar value that defines the learning rate during the gradient descent. Default value is 0.003. A larger rate can drive every unit of a hidden layer negative, after which a rectifier passes no gradient and the network stops training. Applies only when 'Solver' is 'sgd'.
'Solver'A character vector naming the solver that trains the network, either 'lbfgs' or 'sgd'. The default is 'lbfgs', which minimizes the loss over the whole training set at once by limited-memory BFGS, as MATLAB does. It takes no learning rate, stops on the three tolerances below, and reaches a lower training loss in fewer passes over the data, though each of its iterations costs several passes where an epoch costs one. 'sgd' visits the samples one at a time and steps down the gradient of each, running for 'IterationLimit' epochs; it was the default before version 1.9.0.
'GradientTolerance'A nonnegative scalar. Training stops once the gradient’s infinity norm falls to or below it, which is the quantity MATLAB tests too. The default is 1e-6. Applies only when 'Solver' is 'lbfgs'.
'StepTolerance'A nonnegative scalar. Training stops once the step’s infinity norm falls to or below it, which is the quantity MATLAB tests too. The default is 1e-6. Applies only when 'Solver' is 'lbfgs'.
'LossTolerance'A real scalar. Training stops once the training loss falls to or below it. The test is on the loss itself and not on its change, matching MATLAB; pass -Inf to switch it off. The default is 1e-6. Applies only when 'Solver' is 'lbfgs'.
'Activations'A character vector or a cellstr vector specifying the activation functions for the hidden layers of the neural network (excluding the output layer). The available activation functions are 'linear', 'sigmoid', 'relu', 'tanh', 'softmax', 'lrelu', 'prelu', 'elu', 'gelu', and 'none'. The default value is 'relu'.
'OutputLayerActivation'A character vector specifying the activation function for the output layer of the neural network. The available activation functions are the same as for 'Activations'. The default value is 'softmax', which makes the returned scores a probability over the classes and trains the network against cross entropy; any other value trains it against the mean squared error.
'IterationLimit'A positive integer scalar that specifies the maximum number of training iterations. The default value is 1000. Under 'sgd' this counts epochs, under 'lbfgs' solver iterations.
'DisplayInfo'A boolean flag indicating whether to print information during training. Default is false.
'ScoreTransform'A character vector defining one of the following functions or a user defined function handle, which is used for transforming the prediction scores returned by the predict and resubPredict methods. Default value is 'none'.

Source Code: fitcnet

ValueDescription
'doublelogit'1 ./ (1 + exp (-2 × x))
'invlogit'log (x ./ (1 - x))
'ismax'Sets the score for the class with the largest score to 1, and sets the scores for all other classes to 0
'logit'1 ./ (1 + exp (-x))
'none'x (no transformation)
'identity'x (no transformation)
'sign'-1 for x < 0, 0 for x = 0, 1 for x > 0
'symmetric'2 × x - 1
'symmetricismax'Sets the score for the class with the largest score to 1, and sets the scores for all other classes to -1
'symmetriclogit'2 ./ (1 + exp (-x)) - 1

Source Code: fitcnet

The weights of each layer are drawn from a uniform range whose half-width is set by that layer’s activation, and the scheme cannot be chosen: a rectifying activation ('relu', 'lrelu', 'prelu', 'elu', 'gelu') takes the He range sqrt (6 / fan_in), because it passes only half of its input, and the remaining activations take the Glorot range sqrt (6 / (fan_in + fan_out)), which accounts for the backward pass as well. A network whose layers do not share an activation is therefore built with both schemes. What each layer was given is reported by the LayerWeightsInitializers field of the fitted model’s ModelParameters.

See also: ClassificationNeuralNetwork

Source Code: fitcnet

  1. Train a network on Fisher's iris data and see what it got right
 load fisheriris
 Mdl = fitcnet (meas, species);
 pred_species = resubPredict (Mdl);
 confusionchart (species, pred_species, 'Title', ...
                 'Neural network classification of Fisher''s iris data');
plotted figure

  1. Watching the fit converge
 load fisheriris
 Mdl = fitcnet (meas, species, 'IterationLimit', 400);

TrainingHistory records what the solver converges on. The default solver is lbfgs, so that is the loss and the gradient norm; under 'sgd' it is the loss and the accuracy instead.

 h = Mdl.TrainingHistory;
 plotyy (h.Iteration, h.TrainingLoss, h.Iteration, h.Gradient);
 xlabel ('Iteration');
 title ('Training loss, left, and gradient norm, right');
plotted figure

  1. Rectified hidden layers train faster than sigmoid ones
 load fisheriris
 iters = [5, 10, 25, 50, 100, 200, 400];
 L = zeros (2, numel (iters));
 for k = 1:numel (iters)
   for a = 1:2
     act = {'relu', 'sigmoid'}{a};
     m = fitcnet (meas, species, 'Activations', act, ...
                  'IterationLimit', iters(k));
     L(a,k) = loss (m, meas, species, 'LossFun', 'classiferror');
   endfor
 endfor
 semilogx (iters, L(1,:), 'o-', iters, L(2,:), 's-', 'linewidth', 1.5);
 xlabel ('Iteration limit');
 ylabel ('Misclassification rate');
 legend ({'relu', 'sigmoid'});
 title ('A sigmoid shrinks the gradient at every layer');
plotted figure

  1. What the network learned, over two predictors
 load fisheriris
 X = meas(:,3:4);
 Mdl = fitcnet (X, species, 'LayerSizes', [12, 12], 'IterationLimit', 400);

Ask about a grid and paint each point by the answer

 [gx, gy] = meshgrid (linspace (0.5, 7.5, 120), linspace (0, 3, 120));
 [~, ~, region] = unique (predict (Mdl, [gx(:), gy(:)]));
 contourf (gx, gy, reshape (region, size (gx)), [1 2 3]);
 colormap (summer);
 hold on;
 gscatter (X(:,1), X(:,2), species, 'krb', 'ox+');
 hold off;
 xlabel ('Petal length');
 ylabel ('Petal width');
 title ('Decision regions of a two-layer network');
plotted figure