Break large streaming archives apart into their component files
There are several existing zip/unzip libraries out there for JavaScript, including adm-zip, node-unzip, and yauzl. They are each built with slightly different goals in mind, and these differences can mean a lot in production. Molecules was designed for the following qualities:
- Memory efficient: The parser only holds as much data as it needs to emit the next entry, then forgets it.
- CPU efficient: The parser yields to the event loop at the earliest available opportunity.
- Forgiving: If an unsupported compression method occurs, or if some but not all of the zip file is corrupted, an error will be printed and the parser will simply move on to the next header.
In particular, molecules was designed for situations where you need to unzip A LOT of files over a network.
- adm-zip: Not streaming.
- node-unzip: Doesn't gracefully handle certain files or errors.
- yauzl: Not streaming.
npm install molecules
var Unzip = require('molecules').Unzip;
var fs = require('fs');
var mkdirp = require('mkdirp');
var path = require('path');
fs.createReadStream('./archive.zip').pipe(Unzip()).on('entry', function(entry, header) {
if(!header.folder) {
console.log('Writing ' + header.filename);
mkdirp(path.dirname(header.filename), function(err) {
if(err) throw err;
entry.pipe(fs.createWriteStream(header.filename)).on('finish', function() {
console.log('Finished writing ' + header.filename);
});
});
}
}).on('finish', function() {
console.log('Extraction complete.');
});
- No CRC32 Checking
- No ZIP64 support
This library started out as a fork of alunny's well-intentioned but unfinished zstream before becoming a complete rewrite. My fork can be found here.