Table of Contents
- Assign the same matrix to all elements of a struct array
- Multiply a 3d matrix with a vector to get a 2d matrix
- Concatenate a cell array of matrices to one 3d matrix
- Find a Row in Another Matrix
- Preallocate a Cell Array of Full Matrices and Sparse Matrices
- Set the value at many (row,column) index pairs in a matrix
- Create new matrix B where each column in A is repeated k times:
- Delete All-Zero Columns or Rows
- Find n smallest values in a matrix and return their indices
- Normalize rows of matrix to sum to 1
- Normalize each row or column to length one
- To free forgotten file handle locks
- Read in text file
- Plot figure to a printable pdf of correct size
- cell2str
- Speed
- Frequently compare times of different functions
- Simple Parallelization
- Good Optimization Package
- General Matlab Speed-up Packages
- Plot a tree with labels on each node
- Convenient File Reading
- Read in Sparse Index:Value matrix
- Convenient Text File Writing of Matrices and String Cells
- Extract Sentences from a Wall Street Journal tree file
- Find out how many matlab licenses are being used.
- Saving Only a Matrix to PNG File
- Saving an image from imshow without a white/grey background
- Fast Word to Number Mapping with Java Map Containers
- Find all dependencies of a function
- Visualizing a Bivariate Gaussian Distribution
- Sort a cell array based on the length of each cell
- Element-wise max of all elements in a matrix
- Subassign a smaller matrix into a larger matrix at a range of indices
- Compute the Probability of a Point being an outlier
- Comments, Suggestions
- Multiply a 3d matrix with a vector to get a 2d matrix
- Concatenate a cell array of matrices to one 3d matrix
- Find a Row in Another Matrix
- Preallocate a Cell Array of Full Matrices and Sparse Matrices
- Set the value at many (row,column) index pairs in a matrix
- Create new matrix B where each column in A is repeated k times:
- Delete All-Zero Columns or Rows
- Find n smallest values in a matrix and return their indices
- Normalize rows of matrix to sum to 1
- Normalize each row or column to length one
- To free forgotten file handle locks
- Read in text file
- Plot figure to a printable pdf of correct size
- cell2str
- Speed
- Frequently compare times of different functions
- Simple Parallelization
- Good Optimization Package
- General Matlab Speed-up Packages
- Plot a tree with labels on each node
- Convenient File Reading
- Read in Sparse Index:Value matrix
- Convenient Text File Writing of Matrices and String Cells
- Extract Sentences from a Wall Street Journal tree file
- Find out how many matlab licenses are being used.
- Saving Only a Matrix to PNG File
- Saving an image from imshow without a white/grey background
- Fast Word to Number Mapping with Java Map Containers
- Find all dependencies of a function
- Visualizing a Bivariate Gaussian Distribution
- Sort a cell array based on the length of each cell
- Element-wise max of all elements in a matrix
- Subassign a smaller matrix into a larger matrix at a range of indices
- Compute the Probability of a Point being an outlier
- Comments, Suggestions
Assign the same matrix to all elements of a struct array
- Use the deal function and []:
- [allGrad(:).Vb] = deal(df_Vb);
Multiply a 3d matrix with a vector to get a 2d matrix
- % size(A) = [ 22 40 10]
% size(v) = [ 1 10]
H = bsxfun(@times,A,reshape(v,[1 1 length(v)]));
H = sum(H ,3);
% size(H) = [22 40]
Concatenate a cell array of matrices to one 3d matrix
- mat3d = cat(3,manyMatrices{:});
Find a Row in Another Matrix
- % check if queryRow is also a row in matrixToSearchIn
[exists location] = ismember(queryRow,matrixToSearchIn,'rows') - bsxfun with eq, would work too
Preallocate a Cell Array of Full Matrices and Sparse Matrices
- User some values insides the zeros function that resembles the size of your matrices
- bigCellArray = cell(largeNumber,1);
bigCellArray(:) = {zeros(1,80)}; - For sparse matrices, we need to approximate how much data they are likely to store
- oneMat = spalloc(numRows,numCols,40*numRows); % we have around at most numUsedCols = 40, each has numRows entries
allMats = repmat({oneMat},totalNum,1);
Set the value at many (row,column) index pairs in a matrix
- A(sub2ind(size(A),rowIndices,columnIndices))=1;
Create new matrix B where each column in A is repeated k times:
- B=A(:,ceil([1:size(A,2)*k]/k));
Delete All-Zero Columns or Rows
- If you want to delete all rows that contain only zeros:A=A(any(A,2),:)
- This sets the matrix to all those rows we want, we can also explicitly delete them:A(~any(A,2),:) = [];
- If you want to delete all columns that contain only zeros:A=A(:,any(A))
Find n smallest values in a matrix and return their indices
- or just sort the entire matrix and get indices of sorted matrix elements[val pos ] = sort(costMatrix(:));
[x,y] = ind2sub(size(costMatrix),pos);
Normalize rows of matrix to sum to 1
- and do not produce an Na N?, if the sum is 0:E = spdiags(spfun(@(x) 1./x,sum(E,2)),0,size(E,1),size(E,1))*E;
Normalize each row or column to length one
- % each row with diag:
E = E*diag(spfun(@(x) 1./x,sqrt(sum(E.^2,1))),0);
% or each column with bsxfun
E= bsxfun(@times,1./sqrt(sum(E.^2)),E);
To free forgotten file handle locks
- fclose('all')
Read in text file
- each line in one char cellt = textread('plant.train','%s','delimiter', '\n');
Plot figure to a printable pdf of correct size
- set dimensions by hand to fit your figure
- set(gcf,'paperunits','centimeters')
set(gcf,'papersize',[18,6]) % Desired outer dimensionsof figure
set(gcf,'paperposition',[0,0,18,6]) % Place plot on figure
print -dpdf myfigure.pdf
cell2str
- function s = cell2str(cellstr,delimeter)
% Richard _at_ Socher - org
% concatenates a cell array of strings into one long string
if nargin==1
delimeter = ','
end
s = [];
for i=1:length(cellstr),
s = [s, cellstr{i}];
if i~=length(cellstr),
s = [s, delimeter];
end
end
Speed
- never grow matrices inside a loop, always pre-allocate (even if you don't know how large it will be) with an upper bound on what you need and just delete zero or unused entries at the end
- Never change a lot of rows in a matrix, instead change columns, since matlab is column major.
Frequently compare times of different functions
- Use it to compare two possible ways to do the same thing tic; for i=1:10000;inv(rand(3,3));end;toc
Simple Parallelization
- if you want to use 4 cores easily:matlabpool open 4
parfor i = 1:length(parameter)
runComplicatedFunction(parameter);
end
Good Optimization Package
- http://www.cs.ubc.ca/~schmidtm/Software/minFunc.html - has L-BFGS etc.
General Matlab Speed-up Packages
Plot a tree with labels on each node
- Abhishek Sharma modified the below file for prettier printing, see his files: Attach:myTreeplot.txt, Attach:makeTreeFromParsed.txt (change to .m after download)
- function myTreeplot(p,labels)
%TREEPLOT Plot picture of tree.
% TREEPLOT(p) plots a picture of a tree given a row vector of
% parent pointers, with p(i) == 0 for a root and labels on each node.
%
% Example:
% myTreeplot([2 4 2 0 6 4 6],{'i' 'like' 'labels' 'on' 'pretty' 'trees' '.'})
% returns a binary tree with labels.
%
% Copyright 1984-2004 The MathWorks, Inc.
% $Revision: 5.12.4.2 $ $Date: 2004/06/25 18:52:28 $
% Modified by Richard @ Socher . org to display text labels
[x,y,h]=treelayout(p,1:length(p)); % RS: The second argument (1:length(p)) ensures that the leaf nodes are printed in their original order in the parent vector
f = find(p~=0);
pp = p(f);
X = [x(f); x(pp); repmat(NaN,size(f))];
Y = [y(f); y(pp); repmat(NaN,size(f))];
X = X(:);
Y = Y(:);
n = length(p);
if n < 500,
plot (x, y, 'ro', X, Y, 'r-');
else
plot (X, Y, 'r-');
end;
xlabel(['height = ' int2str(h)]);
axis([0 1 0 1]);
for l=1:length(labels)
text(x(l),y(l),labels{l},'Interpreter','none')
end
Convenient File Reading
- Just reads a file into an array of cells, each cell holds one line
- function fileLines = readTextFile(fileName)
fid = fopen(fileName, 'r');
fileLines = textscan(fid, '%s', 'delimiter', '\n', 'bufsize', 99900000);
fclose(fid);
fileLines = fileLines{1};
% % % %Example usage:
% fileName = '../data/someFile.csv'
% fileLines = readTextFile(fileName);
% for li = 1:length(fileLines)
% [~, ~, ~, ~, ~, ~, splitLine] = regexp(fileLines{li}, '\t');
% pols = regexp(fileLines{li}, '\[.(\d)\]', 'tokens');
% %...
% end
Read in Sparse Index:Value matrix
- function sparseMat = readSparseFeatures(filename)
% reads in a sparse index:value feature matrix
% format: first line: number of colums,
% remaning lines are index:value pairs
% assumes that feature indices (columns) start at 0 (will add 1)
% by richard@socher.org
fid = fopen(filename, 'r');
fileLines = textscan(fid, '%s', 'delimiter', '\n', 'bufsize', 99900000);
fclose(fid);
fileLines = fileLines{1};
numCols= str2double(fileLines{1});
fileLines = fileLines(2:end);
numRows = length(fileLines);
% not pretty to keep adding these indices/values
% but it's still fast for most files,
% could pre-allocate these and then cut off afterwards
allRows = [];
allCols = [];
allVals = [];
for li = 1:length(fileLines)
indexValuePairs = textscan(fileLines{li}, '%f:%f');
allRows = [allRows; repmat(li,length(indexValuePairs{1}),1)];
allCols = [allCols ;indexValuePairs{1}+1];
allVals = [allVals ;indexValuePairs{2}];
end
sparseMat = sparse(allRows,allCols,allVals,numRows,numCols);
Convenient Text File Writing of Matrices and String Cells
- function writeTextFile(fileName,fileInput,opt)
% writeTextFile(fileName,fileInput,options)
% Prints a matrix to text file, a cell array of strings, or a cell array of cells of strings
% opt.writeFlag =
% 'a' open or create file for writing; append data to end of file
% 'r+' open (do not create) file for reading and writing
% 'w+' open or create file for reading and writing; discard
% existing contents
% 'a+' (default) open or create file for reading and writing; append data
% to end of file
% 'W' open file for writing without automatic flushing
% 'A' open file for appending without automatic flushing
% opt.separator =
% '\n' (default) or ';' or ' ' etc.
% richard _at_ socher .org
if ~exist('opt','var') || (exist('opt','var') && ~isfield(opt,'writeFlag'))
opt.writeFlag='a+';
end
if ~exist('opt','var') || (exist('opt','var') && ~isfield(opt,'separator'))
opt.separator='\n';
end
if isnumeric(fileInput)
% Write the matrix to file with separated
fid = fopen(fileName, opt.writeFlag);
fprintf(fid, [repmat(['%g' opt.separator], 1, size(fileInput,2)-1) '%g\n'],fileInput);
fclose(fid);
elseif iscell(fileInput{1})
fid = fopen(fileName, opt.writeFlag);
for s = 1:length(fileInput)
for w = 1:length(fileInput{s})
if ischar(fileInput{s})
fprintf(fid, ['%s' opt.separator], fileInput{s}{w});
else
fprintf(fid, ['%s' opt.separator], num2str(fileInput{s}{w}));
end
end
fprintf(fid, '\n');
end
fclose(fid);
else
fid = fopen(fileName, opt.writeFlag);
for s = 1:length(fileInput)
if ischar(fileInput{s})
fprintf(fid, ['%s' opt.separator], fileInput{s});
else
fprintf(fid, ['%s' opt.separator], num2str(fileInput{s}));
end
end
if ~strcmp(opt.separator,'\n')
fprintf(fid, '\n');
end
fclose(fid);
end
Extract Sentences from a Wall Street Journal tree file
- % reads in WSJ file in which each tree is on one line
% splits it at spaces,
% matches all words that end with some parentheses
% richard @ socher .org
inFile = '../data/trees/wsj/finalTestTrees.txt'
outFile = [inFile '_justSentences.txt']
lines = readTextFile(inFile);
fid = fopen(outFile, 'w');
for l = 1:length(lines)
[~, ~, ~, ~, ~, ~, splitStrings] = regexp(lines{l}, ' ');
thisSent = [];
for w = 1:length(splitStrings)
s = regexp(splitStrings{w},'([^ \t\r\n\f\v\)]+)\)+$','tokens');
if ~isempty(s)
thisSent = [thisSent s{1}{1} ' '];
end
end
fprintf(fid, '%s\n', thisSent);
fprintf(1,'.')
end
fprintf(1,'\n')
fclose(fid);
Find out how many matlab licenses are being used.
- lmstat -a
Saving Only a Matrix to PNG File
- The commented out print command uses the current figure which needs a GUI. imwrite can save the matrix without a desktop.
- %print('-dpng',[filename '.png'])
imwrite(I,[filename '.png'])
Saving an image from imshow without a white/grey background
- imshow(img)
set(gca, 'position', [0 0 1 1], 'visible', 'off')
print('img.png', '-dpng')
Fast Word to Number Mapping with Java Map Containers
- Uses code above to read a text file with just the words and then each word's line number is the value for that word in a Java Container (its index).
- Then reads in another file of sentences of words and maps the words to numbers.
- words = readTextFile('wordsOfVocab.txt');
wordMap = containers.Map(words,1:length(words));
% for unknown words, the slow way: unkNum = find(strcmp(words,'UNK'));
unkNum = wordMap('UNK')
% load some file
% fileLine contains words separated by spaces
[~, ~, ~, ~, ~, ~, splitLine] = regexp(fileLine, ' ');
sent = zeros(1,length(splitLine));
for w = 1:length(splitLine)
if wordMap.isKey(splitLine{w})
sent(w) = wordMap(splitLine{w});
else
sent(w) = unkNum;
end
end
Find all dependencies of a function
- [list, ~, ~] = depfun('yourFunctionName')
% filter out matlab built-in functions
for i=1:length(list)
if isempty(strfind(list{i},'R2010a'))
disp(list{i})
end
end
Visualizing a Bivariate Gaussian Distribution
- You can define your covariance matrix sigma and see how it changes the shape of the Normal distribution %% Bivariate Gaussian in 2D
[X1,X2] = meshgrid(-2:.01:2,-2:.01:2);
mu=[0 0];
Sigma=[1 0;0 1];
P=mvnpdf([X1(:),X2(:)],mu,Sigma);
P=reshape(P,size(X1));
pcolor(X1,X2,P);
shading interp
%% Bivariate Gaussian in 3D
h=surfc(X1,X2,P);
set(h, 'linestyle', 'none');
alpha(.5);
Sort a cell array based on the length of each cell
- If you want the shortest entries of allSNum to come first [~, Index] = sort(cellfun('length', allSNum), 'ascend');
allSNum = allSNum(Index);
Element-wise max of all elements in a matrix
- max(1,W) = W(ones(size(W))<W)=1;
Subassign a smaller matrix into a larger matrix at a range of indices
% example subMatrix
smiley = zeros(5,5);
smileyPos = [4,1;5 2; 5 3;5 4;4 5;2 2; 2 4]
for p = 1:size(smileyPos,1)
smiley(smileyPos(p,1),smileyPos(p,2)) = 1;
end
% full image
fullImg = zeros(15,15);
% range of positions where subMatrix should be:
r1 = [1:5];
r2 = [11:15];
% example call of function
fullImg = assignSubMatrix(smiley,r1,r2,fullImg);
function fullMatrix = assignSubMatrix(subMatrix,indexBlock1,indexBlock2,fullMatrix)
% Richard at Socher.org
subMatrix=subMatrix' ;
X = indexBlock1(ceil([1:length(indexBlock1)*length(indexBlock2)]/length(indexBlock2)));
Y = repmat(indexBlock2(:), [1 length(indexBlock1)]);
setProd = [X(:) Y(:)];
fullMatrix(sub2ind(size(fullMatrix),setProd(:,1),setProd(:,2))) = subMatrix(:);
smiley = zeros(5,5);
smileyPos = [4,1;5 2; 5 3;5 4;4 5;2 2; 2 4]
for p = 1:size(smileyPos,1)
smiley(smileyPos(p,1),smileyPos(p,2)) = 1;
end
% full image
fullImg = zeros(15,15);
% range of positions where subMatrix should be:
r1 = [1:5];
r2 = [11:15];
% example call of function
fullImg = assignSubMatrix(smiley,r1,r2,fullImg);
function fullMatrix = assignSubMatrix(subMatrix,indexBlock1,indexBlock2,fullMatrix)
% Richard at Socher.org
subMatrix=subMatrix' ;
X = indexBlock1(ceil([1:length(indexBlock1)*length(indexBlock2)]/length(indexBlock2)));
Y = repmat(indexBlock2(:), [1 length(indexBlock1)]);
setProd = [X(:) Y(:)];
fullMatrix(sub2ind(size(fullMatrix),setProd(:,1),setProd(:,2))) = subMatrix(:);
Compute the Probability of a Point being an outlier
%loOP_script
% use this many points to define the neighborhood
kNN = 50;
lambda = 3;
% needs slmetric_pw
addpath(genpath('slmetric_pw'))
load('ex8data1.mat');
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% Compute Local Outlier Probabilities %
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% for data points X (trained mapped images later)
% each point is one row of this matrix
%compute all nearest neighbors
allDist = slmetric_pw(X',X','eucdist');
%first row are just the points, then follow the nearest neighbors
[allPointsNNDistances allPointsNN] = sort(allDist);
S = allPointsNN(2:kNN+1,:);
Sdist = allPointsNNDistances(2:kNN+1,:);
sigma = sqrt(sum(allPointsNNDistances.^2)./kNN);
pdist = lambda * sigma;
% expected value: E_{s\in S(o)}[pdist(s,S(s))]
Epdist = mean(pdist(S));
plof = pdist./Epdist -1;
nplof = lambda * sqrt(mean(plof.^2));
loop = erf(plof./(nplof * sqrt(2)));
loop(loop<0) = 0;
% outlier threshold:
outliers = loop>0.7;
% Visualize the example dataset
fprintf('Visualizing example dataset for outlier detection.\n\n');
hold on
plot(X(:, 1), X(:, 2), 'bx');
axis([0 30 0 30]);
plot(X(outliers, 1), X(outliers, 2), 'ro', 'LineWidth', 2, 'MarkerSize', 10);
hold off
% use this many points to define the neighborhood
kNN = 50;
lambda = 3;
% needs slmetric_pw
addpath(genpath('slmetric_pw'))
load('ex8data1.mat');
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% Compute Local Outlier Probabilities %
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% for data points X (trained mapped images later)
% each point is one row of this matrix
%compute all nearest neighbors
allDist = slmetric_pw(X',X','eucdist');
%first row are just the points, then follow the nearest neighbors
[allPointsNNDistances allPointsNN] = sort(allDist);
S = allPointsNN(2:kNN+1,:);
Sdist = allPointsNNDistances(2:kNN+1,:);
sigma = sqrt(sum(allPointsNNDistances.^2)./kNN);
pdist = lambda * sigma;
% expected value: E_{s\in S(o)}[pdist(s,S(s))]
Epdist = mean(pdist(S));
plof = pdist./Epdist -1;
nplof = lambda * sqrt(mean(plof.^2));
loop = erf(plof./(nplof * sqrt(2)));
loop(loop<0) = 0;
% outlier threshold:
outliers = loop>0.7;
% Visualize the example dataset
fprintf('Visualizing example dataset for outlier detection.\n\n');
hold on
plot(X(:, 1), X(:, 2), 'bx');
axis([0 30 0 30]);
plot(X(outliers, 1), X(outliers, 2), 'ro', 'LineWidth', 2, 'MarkerSize', 10);
hold off
Comments, Suggestions