Tilted Forum Project Discussion Community  

Go Back   Tilted Forum Project Discussion Community > Interests > Tilted Technology


 
 
LinkBack Thread Tools
Old 04-10-2006, 03:58 PM   #1 (permalink)
Insane
 
Bratwurst's Avatar
 
Location: La la land
Looking for a script or batch file to make zip files

Anyone know how to do something like this? Maybe through a batch file?

1. identify all files within a folder (including subfolders) that are over 250MB
2. Run through the directory of files and zip each identified into a .zip file with the same name, including the file extension (ext: myfile.dbf ==> myfile_dbf.zip)
3. verifies that the zip archive is valid
4. delete the original files which was zipped (upon a successful verification in step 3)

ideally i'd like to run multiple occurrences of such a script/app, or have the app/script take advantage of as much horsepower as possible and process multiple files in parallel.
__________________
40
Bratwurst is offline  
Old 04-10-2006, 06:23 PM   #2 (permalink)
Darth Papa
 
ratbastid's Avatar
 
Location: Yonder
Shame you're talking batch file. I could write it for you in thirty seconds in Bash or Perl. Especially Perl, it's like a two-liner. But you probably wouldn't want to install Cygwin just for this...
ratbastid is offline  
Old 04-10-2006, 06:29 PM   #3 (permalink)
Adequate
 
cyrnel's Avatar
 
Location: In my angry-dome.
A few zippers have scripting methods included. I don't know them so resort to ugliness.

Copy to compressbigstuff.bat:
Code:
@echo off
for %%i in (*.txt) do if %%~zi GTR 250000000 7za a dummy -tgzip -so %%i > %%~ni%%~xi.gz && 7za t %%~ni%%~xi.gz && if %ERRORLEVEL% EQU 0 erase %%i
Scary but functional. It only compresses 250MB txt files as is. Assumes 7za (http://www.7-zip.org/) is in your path. (BTW, 7zip is very efficient compared to rar, winzip, etc.)

Change the "(*.txt)" to "*" for everything, or allow passing in your own directory with %1, etc. You could have it follow directories by changing "for" to "for /R", but I wasn't that adventurous. (see "for /?" from the command line)

7zip can be multithreaded if you ask it to be with some compression types. I wouldn't bother forcing multiple instances or it'll be slower from overhead and thrashing, and force file fragmentation.
__________________
There are a vast number of people who are uninformed and heavily propagandized, but fundamentally decent. The propaganda that inundates them is effective when unchallenged, but much of it goes only skin deep. If they can be brought to raise questions and apply their decent instincts and basic intelligence, many people quickly escape the confines of the doctrinal system and are willing to do something to help others who are really suffering and oppressed." -Manufacturing Consent: Noam Chomsky and the Media, p. 195
cyrnel is offline  
Old 04-10-2006, 07:46 PM   #4 (permalink)
Insane
 
Bratwurst's Avatar
 
Location: La la land
Hey guys thanks a ton! I'll try this Cyrnel, and Ratbastid, I don't mind using perl. I have perl installed on a different machine for MRTG so I can probably install it again elsewhere. I'm not very learned in the finer points of it, but I'm sure I can get it working.
__________________
40
Bratwurst is offline  
Old 04-11-2006, 12:46 PM   #5 (permalink)
Insane
 
Bratwurst's Avatar
 
Location: La la land
Quote:
Originally Posted by cyrnel
Assumes 7za (http://www.7-zip.org/) is in your path. (BTW, 7zip is very efficient compared to rar, winzip, etc.)
I installed 7-zip, but I think the above about path is causing an error. I'm getting this error when I try to run it:

M:\test\FL\develop>zip.bat
'7za' is not recognized as an internal or external command,
operable program or batch file.

Do I need to do something further to get this to run?
__________________
40
Bratwurst is offline  
Old 04-11-2006, 12:57 PM   #6 (permalink)
Insane
 
Bratwurst's Avatar
 
Location: La la land
OK I changed 7za to 7zG and it runs now, makes the file for the gz when testing, but 7zip doesn't actually run and create the file as far as I can tell?
__________________
40
Bratwurst is offline  
Old 04-11-2006, 01:03 PM   #7 (permalink)
Adequate
 
cyrnel's Avatar
 
Location: In my angry-dome.
You need the command-line version.

Can't sidetrack to get you the url. I'm at a customer's right now & just snuck a tfp read in. I'll be back soon.
__________________
There are a vast number of people who are uninformed and heavily propagandized, but fundamentally decent. The propaganda that inundates them is effective when unchallenged, but much of it goes only skin deep. If they can be brought to raise questions and apply their decent instincts and basic intelligence, many people quickly escape the confines of the doctrinal system and are willing to do something to help others who are really suffering and oppressed." -Manufacturing Consent: Noam Chomsky and the Media, p. 195
cyrnel is offline  
Old 04-11-2006, 01:27 PM   #8 (permalink)
Insane
 
Bratwurst's Avatar
 
Location: La la land
No problem thanks for the tip - I'll see if I can track it down
__________________
40
Bratwurst is offline  
Old 04-11-2006, 01:44 PM   #9 (permalink)
Insane
 
Bratwurst's Avatar
 
Location: La la land
I just tried it with the cmd line version and it worked! kick ass!
__________________
40
Bratwurst is offline  
Old 04-11-2006, 03:03 PM   #10 (permalink)
Insane
 
Bratwurst's Avatar
 
Location: La la land
Cyrnel, I ran it with the "for /R" and it started going through the sub directories like you said, however, it put the resulting compressed file into the same directory that I ran the batch file from. Is there a way to put a wildcard in there so the files remain in the original directories?

I wish I knew this as well as you guys do.
__________________
40
Bratwurst is offline  
Old 04-11-2006, 04:12 PM   #11 (permalink)
Adequate
 
cyrnel's Avatar
 
Location: In my angry-dome.
Sorry about that. Originally tried your "name-ext.zip" preference and left the baggage to trip over itself.

Code:
for /R %%i in (*.txt) do if %%~zi GTR 250000000 7za a dummy -tgzip -so %%i > %%i.gz && 7za t %%i.gz && if %ERRORLEVEL% EQU 0 erase %%i
BTW, if you look at help for these things it's all self explanatory. At least you'd recognize what's going on. Stringing it together is just a matter of familiarity.

This old DOS stuff isn't something to be impressed by. Go for PERL or other if you have real work to do. It'll mean fewer migraines.
__________________
There are a vast number of people who are uninformed and heavily propagandized, but fundamentally decent. The propaganda that inundates them is effective when unchallenged, but much of it goes only skin deep. If they can be brought to raise questions and apply their decent instincts and basic intelligence, many people quickly escape the confines of the doctrinal system and are willing to do something to help others who are really suffering and oppressed." -Manufacturing Consent: Noam Chomsky and the Media, p. 195
cyrnel is offline  
Old 06-01-2006, 10:36 AM   #12 (permalink)
Insane
 
Bratwurst's Avatar
 
Location: La la land
I'm really sorry I didn't write back sooner on this, I'm just now getting back to this project (been a rough couple months here at work!).

Just ran the script (well it is making its way through the directories I setup for testing as I write this) and it seems to be working well.

Do you guys think it is worth it to run this through Perl? I really don't mind installing it if it would be a lot better. Like I said before, I had Perl installed on a different machine to run MRTG and I am looking to maybe do RRDTool soon too.

Thanks again for the script - I'll let you know how it ends up.
__________________
40
Bratwurst is offline  
Old 06-01-2006, 10:59 AM   #13 (permalink)
Adequate
 
cyrnel's Avatar
 
Location: In my angry-dome.
Almost all of the real work is in zipping so Perl wouldn't necessarily be better for this application. But, you would be learning a tool that works everywhere and has far more power. Batches are good for quick and dirty Windows stuff, and it's always "installed", but they're horrible for complexity or if you need user interaction.

Spend a little time at http://learn.perl.org then convert the above to Perl.
__________________
There are a vast number of people who are uninformed and heavily propagandized, but fundamentally decent. The propaganda that inundates them is effective when unchallenged, but much of it goes only skin deep. If they can be brought to raise questions and apply their decent instincts and basic intelligence, many people quickly escape the confines of the doctrinal system and are willing to do something to help others who are really suffering and oppressed." -Manufacturing Consent: Noam Chomsky and the Media, p. 195
cyrnel is offline  
Old 06-13-2006, 09:45 AM   #14 (permalink)
 
KnifeMissile's Avatar
 
Location: Waterloo, Ontario
Quote:
Originally Posted by ratbastid
Shame you're talking batch file. I could write it for you in thirty seconds in Bash or Perl. Especially Perl, it's like a two-liner. But you probably wouldn't want to install Cygwin just for this...
If you want to go the PERL route (and it's not a bad route to go), you don't need to install Cygwin. Indeed, I've never liked that environment. You can get a version of PERL that's native to Win32, called Active PERL. Now that I'm looking at the page, it appears they have other "active" langauges, like Python and Ruby, which is interesting... Anyway, if you're looking to install Perl for your Windows system, I highly recommend this one! This is the version of PERL that I use, before I learned Ruby...

Which brings me to my next topic, have you considered something a little more powerful than even PERL, like Python or Ruby? I prefer Ruby myself, so I'll advocate that, but Python is just as powerful so I can't fault you for choosing it.
Just giving you some options...
KnifeMissile is offline  
 

Tags
batch, file, files, make, script, zip

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On



All times are GMT -8. The time now is 11:14 AM.

Tilted Forum Project

Powered by vBulletin® Version 3.8.7
Copyright ©2000 - 2020, vBulletin Solutions, Inc.
Search Engine Optimization by vBSEO 3.6.0 PL2
© 2002-2012 Tilted Forum Project

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73