Microarray::DataMatrix::SmallDataMatrix - abstraction to matrices that fit in memory
smallDataMatrix is an abstract class, which provides as abstraction to an in memory matrix of data. It should not be subclassed by concrete subclasses. Instead, the subclass anySizeDataMatrix, can be subclassed with concrete subclasses, which will provide abstractions to dataMatrices stored in particular file formats, such as pcl files.
Internally, all data are read into memory, and the actual data matrix is stored as a 2 dimensional array. The indexes of the array which are still valid are stored in internal hashes, such that subsequent manipulations of the data only consider the data that have not been filtered out. As rows or columns are filtered out by some of the methods, the entries for these rows or columns are deleted from the hashes that track valid data. Thus when data are redumped to a file, only those data that have not been filtered are printed out.
As smallDataMatrix is an abstract class, it has no constructor. However, the subclass, anySizeDataMatrix, once it has determined that a matrix will indeed fit into memory, MUST call the _init method, which will result in all data being read into memory.
This protected method will read in all the data for the matrix, and store it in memory. It MUST be called during initialization of a subclass object, before any other methods can be called by the client.
Usage:
$self->_init;
or:
$self->SUPER::_init;
This private method uses the _dataLine method from a concrete subclass to request all data from the matrix, and read it into memory, then store it internally.
Usage :
$self->__readInAndStoreAllData;
This private method calculates and returns the average value for a column, depending on whether the mean or median was requested.
my $average = $self->__columnAverage($column, 'mean');
This private method centers the data for a single column, by subtracting the average from every valid value.
$self->__centerColumn($column, $average);
This private method filters out rows that do not have a count for some particular property above or equal to a threshold. It accepts a hash reference, that hashes the row number to a count, and a threshold value. Note that not all rows are necessarily entered into the hash, so this method iterates over all rows, and checks each valid one for its count in the hash, then invalidates those with too low a count.
$self->__filterRowsByCount(\%count, $numColumns);
This private method calculates the standard deviations for each valid column, and returns references to two hashes. Both have the column index as the key, and one has the standard deviation as the values, the other has the column means as the values.
mean = Sum of values/n std dev = square root (((n * sum of (x^2)) - (sum of x)^2)/n(n-1))
my ($stddevHashRef, $meansHashRef) = $self->__validColumnsStdDevAndMeanHashRefs($lineEnding);
This private function dies, with an appropriate error message, if any negative data value is found within the matrix.
$self->__dieIfNegativeDataExistInMatrix;
This private setter method receives a reference to an array of array references (which contains the matrix itself), and stores it as a private attribute of the object.
$self->__setMatrix(\@matrix);
This private setter method receives a pointer to a hash of hashes that stores the percentiles of the data. The first key in the hash is the row from which an element of data came, and the second is the column. The value is the percentile for that piece of data in the column in which it is found.
$self->__setPercentiles(\%percentiles);
This private method deletes entries in the percentiles hash that are not needed - it is really just to save memory....
$self->__pareDownPercentiles;
This private mutator method makes a row invalid. The invalidation is actually done by the the super class, but here, to save memory, we delete the data from the in memory matrix itself. This method is not undoable, because the invalidation also deletes the data for the row.
$self->__invalidateMatrixRow($row);
This private method returns a reference to the 2-D array of data owned by the self object.
my $matrixArrayRef = $self->__matrixArrayRef;
Note: These methods provide the backend nuts and bolts for a transformation or filtering. They should only be called by the immediate subclass, anySizeDataMatrix, and not directly by the concrete subclasses of anySizeDataMatrix. In addition, note that the companion bigDataMatrix must (and does) provide identical interfaces to these methods (obviously with different underlying implementations), such that anySizeDataMatrix can call the methods without regard to the size of the underlying matrix.
This protected method centers each column of data, and returns the largest absolute value that was used in the centering. The caller of the method must specify whether to center by means or medians.
$self->_centerColumns('mean', $lineEnding, $numColumnsToReport);
This protected method actually centers the row data, by calculating the average (mean or median, depending on what was requested) for each row, and then subtracting that value from each valid datapoint in the row.
$self->_centerRows('median', $lineEnding, $numRowsToReport);
This protected method invalidates rows that do not have greater than the requested percentage of present data.
$self->_filterRowsByPercentPresentData($percent, $lineEnding, $numRowsToReport);
This protected method invalidates columns that do not have greater than the requested percentage of present data.
$self->_filterColumnsByPercentPresentData($percent, $lineEnding, $numColumnsToReport);
This protected method filters out rows based on their column percentile, when all data are known to be in memory, and optionally allows for the percentiles of each datapoint to be displayed in the output file.
$self->_filterRowsOnColumnPercentile($lineEnding, $numColumnsToReport, $percentile, $numColumns, $showPercentile);
This protected method will filter out rows whose values do not deviate from the column mean by a specified number of standard deviations, in at least numColumns columns.
$self->_filterRowsOnColumnDeviation($lineEnding, $numRowsToReport, $deviations, $numColumns);
This protected method filters out rows whose values do not pass a specified criterion, in at least numColumns columns.
$self->_filterRowsOnValues($value, $method, $lineEnding, $numRowsToReport, $numColumns);
This protected method filters out rows based on whether the vector that their values define has a length of greater than the specified length.
$self->_filterRowsOnVectorLength($requiredLength, $lineEnding, $numRowsToReport);
This method log transforms the contents of the data matrix, using the specified base for the log transformation.
$self->_logTransformData($logBase, $lineEnding, $numRowsToReport);
This protected method scales the data for particular columns as specified by the client, when all data are in memory.
$self->_scaleColumnData($columnsToFactorsHashRef, $lineEnding, $numColumnsToReport);
This method dumps the current contents of the dataMatrix object to a file, either whose name was provided as a single argument, or to a file whose name was used to construct the object.
$self->dumpData($file);
Gavin Sherlock
sherlock@genome.stanford.edu
To install Microarray::Config, copy and paste the appropriate command in to your terminal.
cpanm
cpanm Microarray::Config
CPAN shell
perl -MCPAN -e shell install Microarray::Config
For more information on module installation, please visit the detailed CPAN module installation guide.