matlab - Using parfor with options to parallelize code

Question

I have the following MATLAB code:

n_part = 5; %count the participants
options.params.cutoff=1;
options.params.lags=0;% use zero, to set n=0 as in eq 4 in the paper, otherwise you will include historical values in the model (one connectivity matrix per historical value) 
options.params.autocorrelation=0; % to exclude self dependancies, since we have already take care of this. This makes the diagonal values equal to zero in the model
options.params.concurrent=1; % must be one to include the concurrent information
options.params.pass_th=0;
options.perc=[0.7 0.3]; % First argument is the partition size to calculate SV, second argument is the size of the test sample
options.rep_svd=10; % how many times to repeat the svd decomposition to maximize out of sample data predictions
options.rep_model=1; % if number of frames not empty, how many times recalculate the model based on the minimum required frames

parfor i=1:n_part
    display(['Running participant ' num2str(i)])
    options.min_frames=size(signal,1); %how many frames to include in the model
    options.SV=SV; %assign SV to options
end

I want to optimize the for loop, since each iteration is independent. So, first, I tried this:

parfor i=1:n_part
    .
    .
    .
end

That gave me this error:

Error: The variable options in a parfor cannot be classified.

The error occurs on this line:

parfor i=1:n_part

I then tried this:

parpool(2);
parfor i=1:n_part
    display(['Running participant ' num2str(i)])
    y=TC{i};
    TC_no_AC_LS{i}=remove_autocorrelation(y,n_ar);
    
    frameSV(i).signal=TC_no_AC_LS{i}; % pick the signal with no autocorrelation
    [SV, R]=model_tsvd(signal,options); %calculate the SVD
    
    frameSV(i).min_frames = size(frameSV(i).signal, 1);
    frameSV(i).SV = SV;
end

for i = 1 : n_part
    options.min_frames= frameSV(i).min_frames; %how many frames to include in the model
    options.SV = frameSV(i).SV; %assign SV to options
    signal = frameSV(i).signal;
    m(:,:,i) = make_model_tsvd(signal, options); % calculate the model
end

But got this error:

Starting parallel pool (parpool) using the 'local' profile ...
Connected to the parallel pool (number of workers: 2).
Error using connectotyping_example2>(parfor supply)
Undefined function 'frameSV' for input arguments of type 'double'.

Error in connectotyping_example2 (line 27)
parfor i=1:n_part

Is there a way to fix this, so the iterations of the loop can run in parallel?

question from:https://stackoverflow.com/questions/65941634/using-parfor-with-options-to-parallelize-code

Answer 1

The issue is the following two lines:

options.min_frames=size(signal,1); %how many frames to include in the model
options.SV=SV; %assign SV to options

Parfor loops must ensure that each loop iteration is independent of the next. In your code above, it looks like options is updated on each iteration, and the updated struct is then used on the next iteration. This cannot be done using parfor, since there is technically no guarantee that iteraion i and i+1 will be done on the same machine.

If min_frames and SV are not required to be a part of the options struct, you could store their value on each iteration in a data-independent way:

parfor i=1:n_part
    % ...
    frameSV(i).min_frames = size(signal,1);
    frameSV(i).SV = SV;
end
% Do some calculation with frameSV to get the data you need, after the loop

This will create a 1 x n_part struct array where the values of min_frames and SV from each loop iteration are stored independently.

Again, this is all assuming that the value of options does not need to be updated on one iteration and used on the next.